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© Vision system and method for automated painting equipment 



© A machine vision system (101; 102) used with 
automated painting equipment (30; 103; 104) for 
object identification, intrustion-sensing and clearance 
monitoring. The vision system includes one or more 
cameras (111; 113; 114) mounted at convenient 
clean locations where unique or distinctive visual 
attributes or features of the objects to be painted, 
such as automotive vehicle bodies (61; 62; 63) or 
parts (230, 236. 238; 240, 246, 248) thereof, and the 
automated painting equipment may be seen. The 
camera acquires successive images of a first scene 
(260; 310, 311; 380, 382) which includes at least a 
first movable object (61; 62; 63; 330) to be identified. 
Image processing steps performed on each image 
from the camera include recognizing selected picto- 
rial attributes or features of the object and compar- 
*Jing the recognized attributes of the first object with 
" corresponding reference data from a set of reference 
CO information about the object previously stored in the 
29 system. When a match between the reference data 
and the recognized attributes occurs, the vision sys- 
^"tem provides signals to programmable controllers 
2(103, 104) which operate the automated painting 
equipment (30, 156, 157). Portions of the visual 
©information acquired and/or generated by the vision 
system are selectively displayed upon one or more 
IUCRT monitors (118; 138;* 153) so that an 
operator/observer can verify correct operation of the 
vision system. 
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VISION SYSTEM AND METHOD FOR AUTOMATED PAINTING EQUIPMENT 



FIELD OF THE INVENTION 



This invention relates to machine vision ap- 
paratus, image analysis systems and methods for 
automated painting equipment 

BACKGROUND OF THE INVENTION 



In recent years, general purpose machine vi- 
sion systems utilizing one or more television (TV) 
cameras and a high-speed digital processing sys- 
tem for image analysis have become available for a 
variety of image analysis applications. These vision 
systems typically include a variety of standard 
hardware and software features to perform various 
processing steps including image acquisition, fea- 
ture enhancement image segmentation, feature ex- 
traction and recognition of objects, and features 
and attributes thereof, the attributes of the objects 
or features which can be recognized or determined 
include location with a scene or buffer, area, cen- 
troid of the area, and orientation of the object of 
feature under scrutiny. These general purpose ma- 
chine vision systems must be adapted and tailored 
to a specific application before they can serve any 
useful purpose in an industrial process setting. This 
is accomplished by proper selection and placement 
of vision hardware, which includes locating one or 
more cameras and providing proper lighting and 
camera filters (if needed), creating an applications 
software package or program to direct and control 
the operation of the vision system hardware, and 
interfacing the vision system to the electronic con- 
trollers operating or monitoring the process equip- 
ment 

Automated painting equipment, such as paint 
lines for automotive vehicle bodies and parts found 
around the world in the factories of automobile and 
truck manufacturers, presently utilize a significant 
number of photocell devices to identify objects and 
conditions so that the equipment may be properly 
operated in a safe and desired manner. For exam- 
ple, several photocell devices are typically used 
singly or in combination to automatically identify or 
verify the type of object approaching a paint spray 
station or booth within an automated paint line. 
Photocells are also often used to detect the intru- 
sion of a human being into various zones of the 
paint line. They are also used to detect the arrival 
of a vehicle body at a predetermined loction within 
the paint spray station or booth, which thus initiates 



a "zone start" or similar signal, that causes the 
applicators to begin spraying paint. As a final ex- 
ample, they are mounted on and move with mov- 
able paint bell support fixtures or platforms to en- 

5 sure that a proper distance or minimum clearance 
will maintained between the paint bells on the 
fixture and the object or vehicle body being paint- 
ed. In electrostatic paint spraying equipment such 
nozzles are normally referred to as bells, to distin- 

w guish them from the conventional paint spray noz- 
zles which work without the benefit of electrostatic 
forces upon the paint droplets emanating from the 
nozzles. 

Up until the present invention, the photocell 

15 devices have been preferred for object identifica- 
tion, detection and sensing for automated painting 
equipment. They are preferred because they are 
non-contact devices which do not scratch the ob- 
ject (i.e., vehicle body or part) to be painted, are 

20 relatively reliable if properly maintained, and can 
be located so as to not interfere with the movement 
of the objects to be painted through the paint line. 
Yet, photocell devices have drawbacks. The 
photocell devices are easily fooled or disabled by 

25 mischievous workers or by environmental factors, 
such as dirt, dust, paint mist or splatter fogging 
over or otherwise obscuring the transparent protec- 
tive covers of the photocells. Typically they must 
be cleaned often, perhaps as often as three times 

30 per eight hour work shift, in order to ensure that 
they will operate properly. Also, the photocell de- 
vices are rather expensive, particularly when the 
custom-designed placement installation and hook- 
up costs (such as conduit runs and wiring) are 

35 included, with some estimates being as high as 
1000.00 per photocell device. The photocell de- 
vices, if not critically aligned, often times will not 
properly identify an object. Moreover, the object to 
be identified must itself typically be precisely posi- 

40 tioned upon its pallet or other holder that is nor- 
mally carried along by a conveyor system, in order 
to be properly identified. The photocell devices 
may well incorrectly identify or not be able to 
identify an object if the object is slightly out of 

45 alignment upon its pallet or holder. Finally, the 
typical photocell device used on an automative 
paint line has been. estimated to add approximately 
one foot to the longitudinal dimension of such paint 
line, principally on account of the unobstructed, 

so uncluttered space required for a clean or 
interference-free zone for the line of sight of the 
photocell beam. Thus, using photocell devices 
adds materially to the overall size and cost of the 
paint line. 

In order to eliminate most (if not all) of the 
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drawbacks asoclated with the use of photocells on 
paint lines and to permit the paint lines to be made 
physically smaller, I investigated the use of ma- 
chine vision systems as a replacement or alter- 
native for photocells. My work in this area led to s 
my invention described below which offers nu- 
merous advantages over conventional photocell de- 
vices and associated methods for detecting and 
identifying objects and person entering or passing 
through automated painting equipment such as ve- to 
hide body paint lines. My work clearly indicates 
that machine vision systems with one or more 
television-type cameras can be used with painting 
equipment to: (1) replace photocells in many ap- 
plications on paint lines; (2) increase equipment 75 
reliability; (3) reduce maintenance requirements; (4) 
improve the accuracy of the identification of ob- 
jects and persons; (5) reduce set-up or change- 
over costs when objects to be painted having a 
different configuration (such as a new vehicle body 20 
style); (6) increase tolerance to misalignment of the 
objects to be painted on their work rests; and (7) 
save a considerable amount of space on the equip- 
ment such as a paint line as well, thus allowing 
more compact (and hence less expensive) paint 25 
lines to be constructed. 



SUMMARY OF THE INVENTION 



In general, my invention relates to use of gen- 
eral purpose machine vision systems and method 
for identification, monitoring, and control purposes 
on automated paint equipment such as automotive 35 
vehicle body paint line. The objects to be painted 
by the paint line may include entire vehicle bodies, 
or vehicle body parts, or automotive components or 
trim. For the identification of vehicle bodies, body 
parts, or other components, cameras may be 40 
mounted at convenient locations where unique or 
distinctive visual attributes or features of the var- 
ious objects to be identified may be seen. For 
example, a camera may be mounted on or near the 
ceiling of a paint line and look down upon objects 45 
to painted as they are transported into the paint 
line by a suitable conventional conveyor system. 
As the objects come into the field of view of such a 
camera, the machine vision system can process 
each image received therefrom in a very short so 
period of time, which can be less than 100 millisec- . 
onds, and through suitable recognition and com- 
parison steps explained hereinafter, can correctly 
identify known objects, selected from as one of 
several choices programmed in the system mem- 55 
ory, with extremely high reliability. One advantage 
which such machine vision systems have over con- 
ventional combination of photocell devices whose 



output signals are interpreted by software resident 
in programmable controllers is that the machine 
vision system may be placed in a learn mode, and 
may taught to recognize a new configuration of an 
object to be painted, such as a new body style, 
with a relatively small amount of effort. Once such 
learning is completed, the machine vision system 
may be put back into operation and will thereafter 
recognize the newly learned new body style, as 
well as previously learned ones. 

In a second aspect of the invention, the ma- 
chine vision system is used on the paint line to 
provide an intrusion-sensing function in areas 
where only certain designated objects are expect- 
ed, and where unwanted or unexpected objects 
provide cause for concern. For example, near the 
entrance to the electrostatic paint spray booth of 
the paint line, a second camera may be mounted 
to the side of the line and observe the entran- 
ceway. The machine vision system can analyze 
successive images from this camera to determine 
whether a vehicle, human being, or unknown object 
is entering the paint booth. When the system deter- 
mines that a vehicle is entering the paint booth, the 
system can be set up to produce a "zone start" 
signal when the vehicle reaches a predetermined 
location. Such a start signal can be used by a 
programmable (or other) controller that actually op- 
erates the painting equipment to begin the normal 
painting sequence for the vehicle body. When a 
human being is detected, the vision system can 
inform the controller of the paint line to enable the 
controller to take appropriate action, or avoid action 
possibly detrimental to the human being in the 
paint booth. Finally, if the presence of an unknown 
object (i.e., not a recognized vehicle body, compo- 
nent or a human being) is detected at the entrance 
of the paint spray booth, the controllers can be so 
informed so that appropriate measures may be 
taken. 

One important advantage of using a vision sys- 
tem to perform this intrusion-sensing function is 
that the length of the conventional silhouette tunnel, 
which is used in conjunction with an array of 
photocell devices to perform an intrusion-sensing 
function, may be greatly reduced. Since the vision 
system is capable of quickly and accurately iden- 
tifying objects, the silhouette tunnel could be 
turned into a relatively thin wall having an opening 
therein slightly larger than, but otherwise corre- 
sponding to, the end profile of the vehicle bodies to 
be passed therethrough, thus shortening the paint 
line by several feet. 

A third aspect of the present invention involves 
the use of a third camera coupled to the machine 
vision system to monitor the clearance between the 
object being painted and electrostatic paint spray 
bells from which the paint emanates, and the plat- 
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form upon which the bells are mounted. If. for 
example, the bells in question are part of an 
adjustable-elevation roof-mounted machine in an 
electrostatic spray booth which is painting the ve- 
hicle body from above, the clearance between the 
vehicle body and paint spray bells may be deter- 
mined by a camera placed to one side of the paint 
spray booth. With appropriate lighting, this camera 
can observe the paint spray bells and the car body 
in profile, thereby "seeing" the actual gap between 
the paint spray bell and the closest portion of the 
vehicle body. As will explained in detail, a vision 
system processing images received from this cam- 
era can determine the approximate distance or 
amount of clearance between the bells and body, 
and send one or more appropriate signals to the 
programmable controller which controls the relative 
elevation of the roof machine to ensure that appro- 
priate clearances are maintained at all times be- 
tween the bell and car body. This approach has 
several advantages over the prior art practice of 
mounting one or more photocell devices on the 
roof machine itself so that the beams of light from 
the photocell devices are broken before the paint 
spray bell comes too close to the vehicle body 
being painted. First, the camera monitoring this 
clearance distance may be mounted away from the 
roof machine in a much cleaner location, a good 
distance from the paint spray. Second, it provides 
far more information about the actual clearances 
than between the bells and car body one or several 
photocell devices. Third, it may be used to actively 
control the spacing being the bells and object 
being painted, thus optimizing spacing for the best 
possible painting results. 

The foregoing introductory overview of three 
purposes to which my invention may be put should 
enable the following summary of the distinguishing 
features of my vision system to be better under- 
stood. According to a first aspect of my invention, 
there is provided vision apparatus for automated 
painting equipment, comprising: (a) at least one 
camera means for acquiring successive images of 
a first scene from a first portion of the painting 
equipment including at least a first movable object 
to be identified via selected pictorial attributes 
thereof; and (b) an image analysis system for pro- 
cessing successive images received from the cam- 
era means. The image analysis system will nor- 
mally include: (1) means for recognizing selected 
pictorial attributes of the first object when such 
attributes are present in at least one of the succes- 
sive images, (2) means for comparing the recog- 
nized attributes of the first object with correspond- 
ing reference data from a first set of information 
previously stored in the system, and (3) means for 
identifying the first object when a match within a 
predetermined first tolerance exists between at 



least one of the recognized pictorial attributes and 
corresponding reference data from the first set of 
information. The vision apparatus may further com- 
prise either or both of the following: means for 

5 communicating the identification of the first object 
as being present in the first scene to the auto- 
mated painting equipment; and means for visually 
communicating the identification of the first object 
as being present in the first scene to a human 

io being. The later means may be a two dimensional 
image display means for displaying at least part of 
the scene containing the object, such as a cathode 
ray tube ("CRT"), i.e., a TV-type monitor. 

To provide for a zone start signal as described 

75 above, the image analysis system of the vision 
apparatus may further include: means for determin- 
ing when the first object has moved into the scene 
sufficiently to have reached a predetermined loca- 
tion within the first scene; and means for sending a 

20 signaJ to the painting equipment when the first 
object has reached the predetermined location 
within the first scene. Also, the image analysis 
system may further include means for recognizing 
the entry of a human being into the first scene. 

25 To provide for monitoring of the clearance be- 
tween the paint bell and vehicle body, as described 
above, the vision apparatus needs the ability to 
identify a second movable object in the first scene 
by selected pictorial attributes of the second ob- 

30 ject. For this purpose, the image analysis system 
may further include: means for recognizing se- 
lected pictorial attributes of the second object when 
such attributes are present in at least one of the 
successive images at the same time selected at- 

35 tributes of the first object are present therein; and 
means for comparing the recognized attributes of 
the second object with corresponding reference 
data from a second set of information about the 
second object previously stored in the system; and 

40 means for identifying the second object as being 
present in the first scene when a match within a 
predetermined second tolerance exists between at 
least one of the recognized attributes of the second 
object and corresponding reference data from the 

45 second set of information. 

According to another aspect of the present 
invention, there is provided a method of using a 
machine vision system in connection with the op- 
eration of automatic painting equipment, compris- 

50 ing the steps of: (a) acquiring, through the use of at 
least one camera means, successive images of a 
first scene associated with at least a first portion of 
the automated painting equipment, the first scene 
from time to time included therein at least a first 

55 movable object having selected pictorial attributes; 
and (b) processing successive images received 
from the camera means using an image analysis 
system. The processing of Step (b) is preferably 
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done by: (1) recognizing selected pictorial at- 
tributes of the first object within the first scene; (2) 
comparing the recognized attributes of the first 
object with corresponding reference data from a 
first set of information previously stored in the 
image analysis system; (3) identifying the first ob- 
ject as being present in the scene when a match 
within a first predetermined tolerance exists be- 
tween at least one of the recognized attributes and 
the corresponding reference data from the first set 
of information. The foregoing method preferably 
also includes a task definition procedure which is 
performed before either Steps (a) or (b) above. 
This task definition procedure comprises the steps 
of: (c) acquiring an image of a typical first object to 
be identified; (d) placing the image analysis system 
in a learn mode; and (e) instructing the image 
analysis system as to which of the pictorial at- 
tributes of the first object are to be stored as part 
of the first set of information for use during Step 
(b). 

These and other aspects, objects and advan- 
tages of the present invention will be more fully 
understood by reference to the following detailed 
description taken in conjunction with the various 
figures and appended claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 



The drawings from an integral part of the de- 
scription of the preferred embodiments and are to 
read in conjunction therewith. Identical reference 
numerals designate like components in the dif- 
ferent figures, where: 

Figure 1 is a combination drawing including 
a simplified plan view of a portion of an automotive 
body paint line which shows the general location of 
three TV cameras and includes an electrostatic 
paint spray booth, and a block diagram for two 
machine vision systems and two electronic control- 
lers for the paint line; 

Figure 2 is a block diagram of a preferred 
embodiment of the machine vision system hard- 
ware used in Figure 1; 

Figure 3 is an end elevational view of the 
front portion of the Figure 1 paint line taken along 
line 3-3 of Figure 1 showing the overhead camera 
used to identify each vehicle transported by a 
conveyor system through the paint line; 

Figure 4 is a simplified plan view of a first 
truck body, namely a pick-up truck body with a 
seven-foot-long bed, which shows the four post 
holes in the truck body to be observed by the 
camera in Figure 3, as well as the genera! outline 
of the truck body; 



Figure 5 is a simplified plan view of a sec- 
ond truck body, namely a pick-up truck body with a 
six-foot-long bed, which shows the four post holes 
in the truck body to be observed by the camera in 
5 Figure 3, as well as the general outline of the truck 
body; 

Figures 6 and 7 are simplified plan and side 
views respectively of a third truck body, namely a 
square-back truck body, which shows the two side- 
10 door windows to be observed by the camera in 
Rgure 3. as well as the general outline of the truck 
body; 

Rgure 8 is a simplified front view of the 
screen of a CRT display showing a simplified plan 

75 view image of the seven foot pick-up truck of 
Rgure 4, with graphical symbols and words added 
to the image by the machine vision system of the 
present invention; 

Figure 9 is an enlarged fragmentary top view 

20 showing one of the post holes of the seven foot 
pick-up truck body shown in Rgure 4; 

Rgure 10 is a graph showing the raw gray 
scale ("RGS") level of the pixels along the X axis 
at a selected Y axis elevation of Figure 9, which 

25 shows the abrupt decrease in gray scale level 
within the post hole; 

Rgure 11 is a simplified, fragmentary side 
elevational view of the Rgure 1 paint line showing 
the front end of a truck body emerging from the 

30 silhouette tunnel of the paint line, which generally 
corresponds to the area indicated by line 11-11 in 
Rgure 1, about the field of view of the second 
camera; 

Rgure 12 is a side elevational view of the 

35 same area depicted in Rgure 11, but showing a 
human being emerging from the silhouette tunnel; 

Rgure 13 is a simplified, fragmentary side 
elevational view of the Figure 1 paint line taken 
along line 13-13 showing the roof machine within 

40 the paint spray booth in position to paint the hood 
of a pick-up truck body; 

Rgure 14 is an end elevational view taking 
along line 14-14 of Rgure 1 showing the roof 
machine and pick-up truck body in the same posi- 

45 tion as in Rgure 13; 

Rgure 15 is a side elevational view showing 
the roof machine depicted in Figures 13 and 14 in 
position to paint the inside of the bed of a six foot 
pick-up truck body; 

so Figure 16 is a simplified front view of the 

upper two-thirds of the screen of a CRT display 
showing a simplified side view image, from the 
third camera in Rgure 1, of a pick-up truck body 
wherein the paint bell of the roof machine is too 

55 close to the roof of a pick-up truck; and 

Rgure 17 is simplified front view of the up- 
per two-thirds of the screen of a CRT display 
showing a simplified side view image from the third 
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camera in Figure 1 of a pick-up truck body, show- 
ing three concentric zones about the paint bell and 
a fourth circular zone near the front edge of the 
roof machine. 



DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 



A. The Paint Line 

Referring now to Figure 1 there is shown a 
plan view of automated painting equipment, namely 
a portion of an automotive vehicle body paint line 
30, which includes five substantially enclosed bays 
or sections 32, 34, 36, 38 and 40 separated by 
interior walls 42, 44, 46 and 48 and an entrance 50. 
The paint line 30 may be of any conventional 
design and construction familiar to those in the 
automotive vehicle body and vehicle parts and 
accessories painting industry. Such a paint' line 30 
typically has a conveyor system 54 extending 
therethrough provided with a belt conveyor 56 or 
the like which moves through the central longitudi- 
nal area of the paint line 30 in one direction such 
as is indicated by arrow 58 to transport vehicle 
bodies having a plurality of different body styles, 
such as bodies 61. 62 and 63. therethrough. The 
various vehicle bodies are typically supported upon 
pallets which ride upon the belt conveyor 56 in 
known manner so that the pallets and vehicle bod- 
ies thereon can be stopped if desired by blocking 
movement of the pallet with a mechanical stop or 
dog while the conveyor continues to move. When 
the conveyor 56 is moving and a pallet is not 
mechanically stopped, the pallet will move at the 
exact same speed as the conveyor 56. 

An open (i.e., non-enclosed) vehicle identifica- 
tion (ID) section or area 66 preceding section 32 of 
the paint line 30 also forms part of the paint line. At 
vehicle ID section 66 vehicle bodies being trans- 
ported by the conveyor are temporarily stopped so 
that the vehicle body style may be positively iden- 
tified as will be shortly explained. The sections 32- 
40 and 66 of the paint line 30 are not necessarily 
shown to scale. Each section is dedicated to per- 
forming a different function. For example, in section 
32, automatic dusting equipment (not shown) such 
as conventional feather dusters may be positioned 
to clean off any dust or other fine particles which 
may be present on the vehicle bodies. If desired, 
section 32 may be enlarged and room provided for 
workers with tack cloth to further clean the vehicle 
bodies that are to be painted. The next section 34 
of the paint line 30 may be a manual (or auto- 



mated) spray area where persons (or robots) ar- 
med with compressed-air-powered paint spray 
wands and the like paint difficult-to-reach areas of 
the vehicle. Shown as part of section 34 is a 

5 conventional silhouette tunnel 68 through which the 
vehicle bodies must pass in order to enter section 
36, which is an electrostatic paint spray booth. 
Within the booth 36 are shown nine paint spray 
bells 71 through 79. Side bells 71-73 paint the 

;o outer left side of the vehicles, while side bells 74- 
76 paint the outer right side of the vehicles. Bells 
77-79 are located on a movable platform 80 which 
is part of a ceiling-mounted roof machine 81 (not 
shown in Figure 1) and which may be lowered and 

75 raised as necessary to provide the proper clear- 
ance between the vehicle. (The platform 80 itself is 
sometimes referred to as the "roof machine".) The 
paint spray bells 71 through 79 typically turn on 
and off under a timed sequence which is synchro- 

20 nized with movement of the belt conveyor 56, and 
therefore with the movement of each vehicle body, 
through the paint spray booth 36 in a manner well 
understood by those in the art. 

Section 38 of paint line 30 may be used if 

25 desired for automatic reciprocating paint spray 
equipment or for a second manual paint spray 
operation. Similarly, section 40 of the paint spray 
line 30 may be used for any conventional purpose 
such as paint drying equipment. 

30 In a conventional paint spray line, various 
groups of photocell devices may be used for the 
several purposes described above in the back- 
ground portion of this specification. A few such 
photocell installations are illustrated in Figure 1 for 

35 comparison purposes. For example, a group 82 of 
three photocell devices 83. 84 and 85, each having 
a beam (such as beam 86 represented by a dotted 
line) which projects across the conveyor system 54 
might be used to identify or verify the vehicle body 

40 style on a pallet of the conveyor 56 as it passes 
through section 66 prior to entering dusting section 
32. Similarly a second group 92 of three photocells 
93-95 may be located within the silhouette tunnel 
68 to detect the presence of a vehicle body or 

45 human being, and if a vehicle body is detected, 
cause a zone start signal to be produced at the 
appropriate time in order to synchronize the se- 
quence of operation of the paint bells 71-79 and * 
roof machine 80 with the movement of the vehicle 

50 body through the electrostatic paint spray booth 
36. Finally, a third group of photocell devices (not 
shown) can be mounted just below and move with 
bells 77-79 so as to project a pair of beams trans- 
versely to arrow 58, one beam in front of and one 

55 beam in back of the bells 77-79, to make sure the 
bells are not getting too close to the object they 
are painting. The present invention renders the 
need to use the groups 82 and 92 of photocell 
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devices, and the photocell devices on the roof 
machine 80 unnecessary, since machine vision 
systems with suitable cameras can perform these 
same functions, with ease, reliability and accuracy. 



B. Vision System Hardware and General Software 

The lower half of Figure 1 shows a very simpli- 
fied block diagram of two vision systems 101 and 
102 interconnected respectively to two program- 
mable controllers 103 and 104 which operate the 
paint line 30. Vision system 101 is connected to a 
first camera means 111 by an appropriate cable 
112. Vision system 102 is connected to two cam- 
eras 113 and 114 by cables 115 and 116. The 
lenses of cameras 111, 113 and 114 may be pro- 
vided with polarizing filters to cut down on glare. 
Each camera cable typically includes at least one 
wire for a sync signal from the vision system, and 
one or more wires for the video signal from the 
camera. The vision system 101 includes vision 
system hardware 117, which will be further de- 
scribed with respect to Figure 2, and a two dimen- 
sional image display means such as a CRT 118 
and keyboard input means 119 connected by ca- 
bles 120 and 121. The first vision system 101 also 
includes a signal-conditioning buffer unit 122 
(hereinafter interface board) interconnected to hard- 
ware 117 by a databus 123 to facilitate commu- 
nication through various conventional signal paths 
125 which may include discrete signal cables 126- 
128 carrying signals from the interface board 122 
to the programmable controller 103, and signal 
cables such as 130 carrying signals from the con- 
troller 103 to vision system 101. 

The second vision system 102 similarly in- 
cludes vision system hardware 137, a CRT 138 
and keyboard 139 connected by cables 140 and 
141. The system 102 also includes an interface 
board 142 to facilitate communication along signal 
path 145 between the vision system 102 and prog- 
rammable controller 104, including signal paths 146 
through 149 carrying discrete signals from system 
102 to controller 104 and signal paths 150 and 151 
for carrying signals from controller 104 to system 
102. 

The programmable controllers 103 and 104 
may be programmed in conventional fashion to 
control the operation of the paint line 30.. The broad 
arrows 156 and 157 represent the many wiring 
interconnections between the input/output channels 
of each programmable controller and the input de- 
vices (not shown) such as limit switches, photocell 
devices, pushbuttons, selector switches and the 
like, and output devices (not shown) such as sole- 
noids (to operate hydraulic, pneumatic, paint, and 
solvent valves), motor starters, pilot lights, audio 



alarms and the like used with the paint line 30. 
Broad arrow 158 represents interlocks and commu- 
nications between controllers 103 and 104. 

The vision system 102 also is shown with an 

5 optional CRT 153 connected to hardware 137 by 
cable 154. One or more optional CRTs may be 
provided to allow the operation of the cameras 1 1 3 
and 114 (or 111) associated with vision system 103 
(or 101) to be monitored in different places by 

ro human operators attending the paint line 30 when 
the line 30 is running. 

Figure 2 is a block diagram of a preferred 
embodiment of the vision system hardware 117 (or 
118) for vision system 101 (or 102). which for 

75 convenience may be called the image analysis 
hardware system 170. The hardware illustrated in 
Figure 2 may be purchased from the International 
Robomation/lntelligence Company ( n IRI") of Carls- 
bad, California. The system 170 corresponds to two 

20 models of vision systems provided by that com- 
pany, namely the PX/VR vision system and the 
DX/VR vision system. Each model is a stand-alone 
gray scale image system of extended, high perfor- 
mance and high variable resolution. The model 

25 PX/VR is a production system specifically designed 
for industrial real-time applications. Color graphics 
is a standard feature of both of the models, and 
each model system includes up to eight camera 
inputs, RGB color outputs and bit-mapped graph- 

30 ics. The DX/VR model is a UNIX-based multi-user 
program development workstation that contains the 
hardware resources (including, memory manage- 
ment and 40 Mbyte Winchester disk) needed to 
run a UNIX-like operating system. Each of these 

35 models wili support a high resolution option 
("HRO") that allows image acquisition resolution 
(i.e., camera resolution) and an image processing 
resolution of 1024 by 1024, 512 by 512 or 256 by 
256 pixels. In my two prototype vision systems of 

40 the present invention, I found it unnecessary to use 
the high resolution features of either of these IRI 
vision systems, since the distinguishing features of 
the objects within the scenes being examined 
could be discerned quite satisfactorily with a cam- 

45 era resolution of 256 by 256 pixels. Nevertheless, 
those in the art should appreciate that the high 
resolution option of the variable resolution vision 
system shown in Figure 2 could readily be adapted 
to the present invention should higher resolution 

so images be required for accuracy or to be able to 
discern smaller features sizes. 

The system 170 includes a camera 
interface/digitizer 172, an image input buffer and 
CRT controller 174, a systolic array-based copy 

55 processor 176, a processing buffer and CRT con- 
troller 178, a group 180 of dedicated "iconic" (i.e. 
low-level image) processors, an optional multifunc- 
tional array processor 182, a host computer 184. a 
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direct memory access ("DMA") bus 186. a system 
bus 188. a VME bus communications interface 190 
for communicating with VME bus 192 to which 
other independent computer systems may be con- 
nected, one or more 5 and 1/4 inch floppy disks 
drives 194. a larger hard disk drive 196 (such as a 
40 Mbyte Winchester disk drive), and an optional 
memory controller 198 (to accommodate additional 
memory if needed) all interconnected generally as 
shown. The processing buffer portion of block 178 
includes four frame buffers, each having a size of 
256 x 256 pixels. Without HRO, the PX/VR and 
DX/VR machine vision systems from IRI will accept 
up to 8 RS170 standard TV camera inputs, which 
may be either black or white ( n B/W M ) or color. In 
the prototype systems of the present invention. 
B/W camera signals were found to produce sat- 
isfactory results, even though the vehicle bodies 
were being painted with one (or two) of 36 widely 
different colors, including white, silver, dark blue 
and black. Without HRO, the resolution offered for 
each camera is 256 by 256 pixels, with 256 raw 
gray scale ("RGS") levels. After the B/W TV sig- 
nals input along signal path 200 has been con- 
ditioned and appropriately digitized into one 8 bit 
byte per pixel by camera interface/digitizer 172, the 
signals are passed along through data path 202 to 
the image input buffer and CRT controller 174 
which has 1 Mbyte of random access memory. The 
buffer 174 can be accessed at a sequential ad- 
dressing rate of 40 million pixels per second or a 
random address rate of 12.5 million pixels per 
second. The copy processor 176 can transfer in- 
formation from the image buffer 174 to the DMA 
bus 186 at the rate of up to 12.5 million pixels per 
second. Block 174 also includes a CRT controller 
which can directly output the contents of the image 
buffer in block 174 to any CRT monitor via data 
path 204. 

The host computer 184 may be constructed 
around a Motorola MC86010 processor with no-wait 
states and a 12 MHz clock speed. The processor 
of computer 184 addresses a 512 K byte random 
access memory and an optional 128 Kbyte eras- 
able programmable read only memory (EPROM). 
The DMA bus 186 and its associated controller is 
part of computer 184 and is implemented using a 
Motorola MC68450 four-channel DMA package. 
The computer 184 also includes four RS232 (or 
RS422) ports indicated by lines 208 for serial I/O 
communications, and a 16 bit parallel I/O port in- 
dicated by data path 210. A second parallel I/O 
(Centronics) port indicated by data path 212 is 
available as a printer port, but may be used for 
discrete parallel I/O communications if so desired, 
as has been done in the prototype systems 101 
and 102 of the present invention. 

The image analysis system 170 is character- 



ized by a multiprocessor architecture wherein the 
various processors are able to communicate via the 
system bus 188, as well through certain other 
designated data paths such as paths 214 through 

5 218. The hierarchy of steps undertaken in the im- 
age analysis process are typically distributed over 
the hierarchy of dedicated processors 176, 180. 
182 and 184 so as to be efficiently matched to the 
image analysis task to be performed. Specifically, 

w the host computer 184 which includes a vary fast 
virtual memory management unit, is supported in 
its tasks by the special coprocessors 176. 182 for 
various image preprocessing and feature extraction 
functions. The high-speed array processor 182 is 

is optional, and is used for feature enhancement, im- 
age segmentation, and feature extraction oper- 
ations. It is a highly piped-lined array which op- 
erates in single instruction-multiple data (SIMD) 
mode. The host computer 184 is capable of per- 

20 forming up to one million instructions per second 
(MIPS); the array processor can perform up to 50 
million arithmetic operations per second (MOPS). 

The dedicated iconic processor 180 provides 
additional high-performance hardware feature ex- 

25 traction functions with the ability to isolate a rectan- 
gular window of variable dimensions within a 256 
by 256 pixel image. Within the selected window, 
the following functions can be performed: monadic 
point transformation, and row, column, or diagonal 

30 histogram or profile calculations. By monadic point 
transformations are meant any arbitrary linear or 
non-linear function involving one variable, such as 
image inversion, a single-level conversion, single- 
level thresholding, multi-level thresholding, sup- 

35 pressing certain ranges of gray scale: histogram 
equalization for contrast enhancement and the like. 
In addition the processing buffer 178 and iconic 
processors 180 via image buffer read path 216 and 
read/modify/write data path 218 can perform the 

40 usual reads/modify/write cycle functions. These 
typically are "translate-and-copy" functions execut- 
ed upon command from the host computer 184, 
wherein an image from within one of the 256 by 
256 frame buffers in processing buffer 178 is trans- 

45 lated according to some transformation function by 
the processor block 180 and then the translated 
image is copied into the same or another frame 
buffer within processing buffer 178. 

The PX/VR system runs under an IRI operating 

so system known as RT/M using one or more applica- 
tion programs developed in the FORTH language 
(on the IRI PX/VR system or the IRI model P256 
vision system) or in the language "C" (on the 
DX/VR system or on the IRI model D256 vision 

55 system equipped with a UNIX-based program de- 
velopment facility). Application programs devel- 
oped on the DX/VR or D256 vision systems can be 
down loaded via the RS232 communications facility 
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or a floppy disk to the PX/VR vision system, and 
executed in real-time by the RT/M operating sys- 
tem. RT/M includes facilities for interrupt handling 
and event-driven execution, file management, an 
interactive execution of a very comprehensive im- 
age analysis function (software) library known as 
the "iconic kernel system" (IKS). The model DX/VR 
vision system runs under a REGULUS operating 
system which is compatible with UNIX operating 
systems and provides a UNIX-like user interface, 
(REGULUS is a software product and trademark of 
Alcyon, Inc., while UNIX is a software product and 
trademark of AT&T Bell Laboratories.) Application 
programs developed on the DX/VR vision system 
can be executed on the DX/VR vision system or on 
the PX/VR vision system. The specifics of the 
operation of the vision system 170 hardware, sys- 
tem software and image analysis function library in 
Figure 2 is well documented in existing IRI system, 
hardware and software manuals and literature, in- 
cluding the following: 

(1) A Primer on Machine Vision and the IRI 
Vision Systems, (1985); 

(2) RT/M Reference Manual, Release 2.1 - 
(Mar. 1986); 

(3) Regulus 4.2D D256 Reference Manual 
Supplement (Aug. 1986); 

(4) IKS Manual (Jan. 1 987); 

(5) D256 and DX/VR Reference Manual - 
(Dec. 1986); 

(6) PX/VR Technical Manual (1986); and 

(7) Preliminary Product Bulletins (Undated) 
for PX/VR Variable Resolution System and for 
DX/VR Variable Resolution System . 

Accordingly, further description of the general 
IRI vision systems, such as system 170, need not 
be given here. Moreover, those skilled in the art of 
machine vision systems are well aware of how to 
program and how to use most image processes or 
analysis functions available in a vision system li- 
brary such IKS. Therefore, a detailed explanation of 
applications programming and machine vision im- 
age processing functions need not be given herein. 

C. The Three Vehicle Body Styles & Identification 
Thereof 

Returning to Figure 1 , it can seen that the three 
truck bodies 61, 62 and 63 are three different 
styles of vehicles. Truck body 61 is a pick-up truck 
having a seven-foot-long (2.1 meters long) bed 
230, chassis 232, a cab section 236 and hood 
section 238. The chassis 232 is partly visible be- 
cause the truck bed 230 is temporarily installed 
backwards upon the chassis 232 and spaced apart 
fro?n the cab section 236 by about three feet (0.9 
meter) for the painting operation. This allows the 



rear wall 239 of the cab to be more easily painted 
than if the bed were placed up close against the 
cab section 236. 

The second truck vehicle body 62 has a bed 

5 240 which is six feet long (1.8 meters long), a 
chassis 242 and a cab section 246 and hood 248. 
Like truck body 61, the bed 240 is rotated 180 
degrees from its normal position and separated 
from the cab 246 by about three feet (0.9 meters). 

;o The third truck body 63 shown in Figure 1 is a 
square-back body style having a rear passenger 
section 250, a front passenger section 256 and a 
hood section 258. 

The first camera 111 in Figure 1 scans a 

75 rectangular area 260 (shown in dotted lines) within 
vehicle ID section 66 in order to Identify the style 
of truck body when each truck body to be painted 
by line 30 is temporarily stopped within rectangle 
260 for this purpose. Figure 3 taken along line 3-3 

20 of Figure 1 shows an end view of the entrance 50 
to the paint line 30, and shows the position of 
overhead camera 1 1 1 centered high above the belt 
conveyor 56 on vertical center line 262. As shown 
in Figures 1 and 3, lighting for the scene viewed by 

25 camera 111 is provided by two conventional flu- 
orescent fixture arrays 266 and 268 which run the 
length of paint line 30 approximately 1 4 to 15 feet 
(4.2 to 4.5 meters) above the floor line 270, and are 
approximately 10 feet on either side of center line 

30 262 as indicated by horizontal dimension lines 267 
and 269. The camera 111 is mounted approxi- 
mately 10 feet (3 meters) above the top plane 270 
of the bed 230 (or 240) of the pick-up truck body 
61 (or 62) as indicated by vertical dimension line 

35 272. Each fixture array 266 and 268 consists of a 
series of eight-foot (2.44 meters) twin-bulb fluores- 
cent fixtures arranged end-to-end. The fluorescent 
light fixtures of arrays 266 and 268 may be pro- 
vided with conventional, translucent cover plates 

40 274 to help diffuse (i.e., evenly spread) the light 
upon the objects being inspected by the camera 
111. 

The chassis 232 of truck body 61 is supported 
by a suitable pallet structure 280 which rides on 

45 top of and is frictionally or otherwise releasably 
engaged by belt conveyor 56 in a conventional 
manner. Grating structure 282 provided beneath 
the belt conveyor 56 provides a convenient means 
for containing paint overspray, solvents cleaning 

so fluids and the like used by paint line 30. 

In Figure 3, the profile of silhouette tunnel 68 is 
shown in dotted lines. The tunnel 68 is typically 
used to provide a profile gate through which only a 
vehicle body or a human being, but not both, may 

55 pass. This helps ensure that no one is in paint 
booth section 36 while it is painting a vehicle body. 
The inner dotted line 286 conforms to the end 
profile of the truck bodies 61 through 63 for this 
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purpose. As shown in Figure 1 . conventional paint 
lines use a group 92 of photocells scanning their 
beams across the tunnel opening, as indicated by 
photocell devices 93-95 in Figure 1 to determine 
whether a vehicle body, human being or unidenti- 
fied object is passing through the silhouette tunnel 
68. A conventional silhouette tunnel may be up to 
nine feet long, and will be further discussed with 
respect to Figures 1 1 and 12 later. 

Figures 4 through 7 provide more detailed 
views of the three different types of truck bodies 
61-63 shown in Figure 1, and illustrate the features 
or pictorial attributes of the vehicle bodies which 
the vision system 101 connected to camera 111 
examines in order to recognize and identify the 
different styles of truck bodies 61-63. Figure 4 is 
simplified plan view of truck body 61 in position 
within field-of-view rectangle 260 for identification 
by vision system 101 through an image received 
from camera 111. The pictorial attributes of the 
seven foot (i.e., 2.1 meters) pick-up truck 61 which 
the system 101 looks for in the images are the four 
small rectangular post holes A, B, C and D located 
near the four comers of the pick-up truck bed 230 
in the right and left side walls 231 and 233 thereof. 
(In the prototype systems of the present invention, 
body 61 corresponds to a 1987 model year Ford 
Ranger pick-up truck with a long bed made by the 
Ford Motor Company.) Specifically, the machine 
vision system looks for the presence of these four 
holes at certain pixel locations within the image, 
and checks their size as well, as will be further 
explained. It should be noted that 256 by 256 pixel 
image lends itself to defining locations in terms of 
a rectangular or Cartesian coordinate system, with 
the X and Y axes being the horizontal and vertical 
axes, respectively. Thus, the center of posthole A 
can be described as being at pixel location (X A , 
Y A ), the center of posthole B is at location (X B , Y B ), 
and so on. 

Figure 5 is a plan view of the six-foot (1.8 
meters) pick-up truck body 62 which also has four 
postholes A', b\ c' and d' located near the cor- 
ners of truck bed 240 the right and left side walls 
241 and 243. (In the prototype systems of present 
invention, body 62 corresponds to a 1987 model 
Ford Ranger pick-up truck with a short bed made 
by the Ford Motor Company.) It is these four 
postholes which the vision system 101 via the 
images from camera 111 recognizes and relies 
upon to identify and distinguish body 62 from bod- 
ies 61 and 63. As in Figure 4, the location of the 
centers of the various postholes A - d' can be 
specified by Cartesian coordinates, such as (Xc, 
Y c ). When truck body 62 is brought into vehicle ID 
section 66 of the paint line 30 by belt conveyor 56, 
a hydraulically-actuated mechanical stop (not 
shown in Figure 1) causes the truck body 62 to 



stop in the same basic location as truck body 61. 
Note that the only difference as far as the vision 
system 101 is concerned, between the two vehicle 
bodies 61 and 62 is the length of the bed sections 

5 230 and 240, and the difference in the longitudinal 
location of postholes B. C versus postholes b\ c\ 
(The size and relative location of postholes A - D in 
comparison to postholes a' - o\ such as the trans- 
verse spacing between holes A and D and A and 

w D , and between holes B and C and holes b' and 
C . are substantially identical.) Accordingly, the vi- 
sion system 101 distinguishes between truck bod- 
ies 61 and 62 by taking notice of the relative 
difference in the longitudinal loction or displace- 

15 ment of holes B and C of body 61 in comparison to 
. that of holes B' and C' of body 62. 

Figures 6 and 7 are a plan view and a side 
view respectively of the third vehicle body, namely 
body 63, which is a square-back design. (In the 

20 prototype systems of the present invention, body 
63 corresponds to the 1987 model year Ford Bron- 
co truck made by the Ford Motor Company.) Vision 
system 111 of Figure 1 recognizes the body 63 by 
examination of two features, namely the rearward 

25 side window openings Q and P respectively lo- 
cated in the left side wail 280 and right side wait 
282 of the rear passenger section 250 of the body 
63. The passenger side wall 282 with a substan- 
tially rectangular window opening P is shown in 

30 Figure 7 along with the passenger door 284, which 
has a somewhat smaller window opening 286 lo- 
cated in it. From overhead camera 111, the window 
openings Q and P appear generally as an elon- 
gated rectangle as best shown in Figure 6. From 

35 this perspective, window opening Q has two short 
edges, namely front side R and rear side T, and 
two. long edges, namely lower outside edge S and 
upper inside edge U. Similarly window opening P 
has two short side edges, R' and T, and two long 

40 edges S' and U . As in Figures 4 and 5, it is 
possible to define the approximate location of the 
edges R - U and R - U in terms of a Cartesian 
coordinate system corresponding to the rectangular 
format of the 256 by 256 pixel image received by 

45 vision system 111 from camera 101. Specifically, 
the short edges R and T lie generally along one or 
a few adjacent vertical columns of pixels crossing 
the horizontal axis at X R and X T respectively. Simi- 
larly, the longer edges S and Q are centered about 

so horizontal rows of pixels crossing the vertical axis 
at Y s and Yy. The vision system 101 recognizes 
window openings Q and P by their relative location 
within the field-of-view or image area 260 and by 
ascertaining that the two windows have approxi- 

55 mately the correct amount of area expected for the 
window openings, as will be further explained. 

Figure 8 is a front view of a CRT display 
showing an image of the seven-foot (2.1 meters) 
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pick-up truck body 61 as displayed by vision sys- 
tem 101 upon monitor 118 for the benefit of a 
human operator or observer. When the vision sys- 
tem 101 has recognized a feature it is looking for 
such as holes B, C and D, it indicates this by a 
suitable graphical symbol surrounding each hole, 
such as circles 287-289 centered about the iden- 
tified features. Any suitable symbol (such as oval, 
rectangle, triangle and the like) or visual effect 
(such as video reverse, blinking video highlighting, 
change of color, color contrast or the like) could be 
used instead of a circle. Once the final feature, 
namely hole A in Rgure 8 has been recognized, 
and all appropriate or desired checks have been 
performed based upon the information related to 
the recognized features, the system 111 may in- 
dicate to the human operator visually, if desired, 
that the vehicle body whose image is being cap- 
tured by camera 111 has been positively identified 
as a particular style of vehicle body. The system 
101 may indicate this by overlaying or displaying a 
message such as message 290 which reads 
"BODY STYLE ID: 7 FOOT PICK-UP" to indicate to 
the operator precisely what body style has been 
identified. Also, the vision system 101 can send an 
appropriate signal via data path 123 to interface 
board 122, which in turn produces an output signal 
on one of its output lines such as line 126 to notify 
the programmable controller 103 that the body in 
scene 260 Is the first body style, namely a seven- 
foot pick-up truck. In a similar manner, the vision 
system 101 can cause interface board 122 to gen- 
erate output signals on lines 127, 128 and 129 
which indicate respectively that a six-foot pick-up 
truck body, a square back truck body, and an 
unidentifiable vehicle body is in position within rec- 
tangular area 260. The interaction between prog- 
rammable controller 103 and vision system 101 
can be synchronized by suitable input signals sent 
from programmable controller 103 via input lines, 
such as line 130 going to the interface board 122, 
which can pass such synchronizing signals along 
data path 123 to vision system 101. 

Rgure 9 is an enlarged plan view of hole D in 
Rgure 4. The speckling In areas 292 nd 293 out- 
side and inside of the side and front walls 233 and 
235 respectively of the bed 230, as well as the 
speckling within the hole D itself, is intended to 
show that areas 292 and 293 and hole D are 
considerably darker than the upper surface of side 
walls 233 and 235. Rgure 10 illustrates this change 
in optical density in a graph of the raw gray scale 
(RGS) level of pixels in a horizontal direction from 
right to left at elevation Y 0 taken by a B/W camera 
111. Specifically, the X axis of Rgure 10 repre- 
sents individual pixels, while the RGS levels on the 
Y axis of Rgure 10 represent optical density on an 
absolute scale ranging from a zero (corresponding 



to completely black) to 255 (corresponding to the 
brightest all-white intensity). These 256 levels of 
gray scale are expressed as in binary form using 
an 8-bit byte. For the lighting conditions described 

s above with respect to Rgures 1 and 3, the light 
level on top plane 270 of a typical truck bed being 
examined by TV camera 111 in the prototype of 
the present invention has a gray scale level (as 
perceived by vision system 101) of between about 

;o 50 and about 110. Rgure 10 shows that gray scale 
level for pixels 1-3 and 13-15 range between 60 
and 70, which corresponds to the darker speckled 
areas 290 and hole D. In contrast, the brighter 
areas on top of the side walls 233 and 235 of bed 

75 230 have a gray scale level of between 90 and 
100, as indicated for pixels 5-11 and 17-19 in 
Rgure 10. Pixels 4, 12 and 17 are pixels where the 
gray scale is in transition between the lower levels 
and the brighter levels. 

20 As shown in Rgures 4, 5 and 9, the holes A - 
D and a' - d' are each substantially square. As 
shown in Rgure 9, the approximate size of the 
hole, as perceived by camera 111, is approxi- 
mately three pixels by three pixels, for an area of 

25 nine pixels. The vision system 101 may be prog- 
rammable to recognize each of the holes by look- 
ing for an appropriate pattern of light-to-dark-to- 
light transitions as shown in Rgure 10 in the three 
rows and columns about the centerline for each 

so holes. For example, the centerlines for hole D 
shown in Rgures 4 and 9 is defined by the Carte- 
sian coordinate pair Xo, Y 0 . Since the truck bodies 
61 and 62 are carried by a pallet containing suit- 
able locating fixtures which position the truck with 

35 fairly good precision, it is possible to specify in 
advance the local region or neighborhoods of pix- 
els which should be searched to determine whether 
the holes A-D or the holes A - D are present. 
Specifically, it is only necessary to examine a 

40 relatively small portion, well under 15%, of the total 
number of pixels from an image taken by camera 
111 in order to determine whether the vehicle body 
currently being examined is either a body style like 
body 61, body 62 or body 63. Since the expected 

45 centerlines for each of the holes in known, the 
vision system 101 can be programmed to examine 
only a small number of pixels centered about the 
expected center coordinates of each hole. For ex- 
ample, in order to find hole D, the system could be 

so programmed to examine a 20 by 20 pixel area 
centered about Cartesian coordinates X Dl Y D , or 
400 pixels. To look for ail four holes of truck bed 
230 would thus require examining only 600 pixels 
( = 4 x 400 pixels) out of 65,536 pixels in the 

55 image (256 x 256 = 65.536). In the prototype of 
the present invention. I have determined that an 
accuracy of no more than plus or minus 5 pixels 
' about the centerlines of each of the holes A - D 
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and a' - D' is sufficient to properly locate and 
identify them "rf they are indeed present all four 
holes on truck bed 230 and 240. Accordingly, only 
an area of 10 pixels by 10 pixels, or 100 pixels 
total, is examined to look for each hole about its 
expected centeriine coordinate pair. Thus, only 400 
pixels out of 65.536 pixels in the image, or less 
than 1 0/ o, need be examined to ascertain that track 
body 61 is present by looking for four holes A - D. 

Referring now to Figures 6 and 7, the manner 
in which the vision system 101 examines an image 
for the presence of window openings Q and P may 
now be further explained. The window openings Q 
and P are optically darker (as indicated by the 
speckling) than the surrounding areas of the side 
walls and roof of the rear passenger compartment 
250. Accordingly, the front and rear window edges 
R and T and side edges S and U represent rapid 
transition regions between light and dark. To detect 
the presence of window opening Q. an image from 
camera 111 only need be examined roughly in the 
area from Xr through X T . and Y s and Y Ut which 
dimensions represent the expected locations of the 
edges R. T, S and U of the window opening Q. In 
the prototype system of the present invention, the 
window openings Q and P are always within in the 
same location of fieid-of-view 260 plus or minus 
five pixels when the body 63 is properly seated 
and located upon its supporting pallet. Window 
openings Q and P each occupy an area of approxi- 
mately an area of approximately 200 pixels (about 
25 - 30 pixels long by about 7 - 8 pixels wide). 
Thus, it can be appreciated that either the relative 
location of the window openings or their size (area) 
or both may be checked to determine whether the 
presence of these window openings are present in 
the image from camera 111. As in the Figure 9 
example, a tolerance may be specified as to pre- 
cisely where the window is located, such as plus or 
minus five pixels or plus or minus 10 pixels. Simi- 
larly, a tolerance may be specified for the area, 
such as plus or minus 20 pixels. These tolerances 
may be stored as part of a set of reference in- 
formation about prototypical body 63 in the soft- 
ware library or other memory of vision system 1 01 . 

Before explaining the precise procedure used 
by which the vision system 101 learns the pictorial 
attributes or features of the truck bodies that are to 
be recognized and how it actually distinguishes 
and identifies the three truck bodies, it is useful to 
discuss shorthand notation for distances between 
centroids (i.e., centers) of identified features such 
as the holes A - D and edges of the window 
openings Q and P. In order to determine distances 
between two features, the existence and location of 
the features must first be determined. The recogni- 
tion of the existence of a feature by the IRI vision 
systems described above can be accomplished 



directly be extracting the feature from the gray 
level distribution of the image by using a histogram 
or profile function, or indirectly by first changing 
the RGS level values into binary values through the 

5 use of a segmentation operation (such as threshol- 
ding or blob labeling) and then extracting the out- 
line of the feature by using a chain encoding func- 
tion (which is a conventional image processing 
technique that codes an image contour as a chain 

w of line segments having horizontal, vertical or diag- 
onal direction) or by using a run length encoding 
function (which is a conventional image processing 
technique used in machine vision systems to de- 
tect lines or other geometric forms). Once the 

15 outline of a feature or object has been determined, 
it is simple matter to calculate its first three mo- 
ments of inertia, namely its area, its centroid, (i.e.. 
the X, Y coordinate pair representing its center of 
gravity), and its angle of orientation. (These three 

20 moments are derived automatically with a single 
call to an IRI library function named 
"MOMENTS".) Also, for simple well-defined 
shapes such as the substantially square holes A - 
D shown in Figure 4, it is a very straightforward 

25 matter to write a program which checks successive 
pixels in successive rows and columns of a window 
area of interest, and calculates the relative location 
of the object and its total area by directly examin- 
ing the gray scale levels of the pixels, like those 

30 illustrated in Figure 10. Also, once the relative 
location of a recognized feature, such as the 
square hole 0, has been calculated, it is also a 
straightforward matter to calculate its centroid. 
Once the centroids of two objects or blobs are 

35 known, the distance between them can be readily 
calculated. For example, the distance D A b between 
the centers of hole A and hole B can be calculated 
by using equation 1: 

40 D AB = (Xa-Xb) 2 + (Y A - Y B )2 0 5 (1) 

where X A , Y A and X B , Y B are the Cartesian coordi- 
nate pairs of holes A and B respectively. In a 
similar manner, the distances D B c. D C d and D A o 

45 may be readily calculated, if desired. 

It is also a simple matter to calculate the dis- 
tance D RT between the front and rear edges of 
window opening Q shown in Figure 6. To do this 
the average X dimension, namely X R of front edge 

so R and the average X coordinate, namely X T , of 
edge T can be calculated. Thereafter, the distance 
D RT can be calculated using Equation 2: 

Drt = X R - X T (2) 

55 

Similarly, the distance between the average Y co- 
ordinates, Y s and Yu, of side edges S and U may 
be calculated as per Equation 3: 
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Dsu = Xs • Xg (3) 

The distance between front and rear edges R and 
T and side edges s' and u' may be calculated in 
a manner similar to that of Equations 2 and 3. 



(1) Learn Mode for Vehicle Body identification 



The learn mode of the vision system 101 with 
respect to identification of body styles may be 
described as follows. (Note: All steps are automati- 
cally performed by system 101 in the sequence 
discussed unless otherwise stated.) Step 1: Wait 
for a seven-foot pick-up truck body 61 to come to 
rest in position within area 260. Step 2: Snap an 
image of body 61 using camera 111. Step 3: Dis- 
play the acquired image on CRT 118. Steg 4: Enter 
interactive mode (by the operator pressing appro- 
priate keys on keyboard 119) so that the operator 
can specify (by moving the "cross-hair" cursor 296 
shown in Figure 8 over each hole and pressing the 
"Enter" or "Return" key) the four holes A-D to be 
identified by system 101 on the truck bed 230. 
When the system 101 recognizes a bounded or 
enclosed feature, such as a hole (or window open- 
ing), a circle is drawn around the feature, thus 
allowing the operator to verify that the system has 
recognized the intended feature. Step 5: Calculate 
and automatically record the X, Y coordinates of 
the centers of each of the identified holes A-D as 
part of the reference data in a first set of informa- 
tion stored in the system software library of system 
101. Step 6: Allow the human operator to type in 
using keyboard 119) the English language iden- 
tification, i.e. "seven-foot pick-up," for the refer- 
ence data for vehicle body 61 . Step 7: Derive the 
area of the holes, and record them as part of the 
reference data stored as the first set of information. 
Step 8: Derive distances D A b. D B c. D C d. and D A o. 
and record them as part of the reference data 
stored as the first set of information. This com- 
pletes the learn mode for the seven-foot (2.1 me- 
ters) pick-up body style. 

The learn mode for the six-foot (1.8 meters) 
pick-up truck body 62 is substantially identical to 
the procedures followed for the seven-foot pick-up 
truck body 61 , and differs only in that holes a' - D 
are being identified and that the X, Y coordinate 
information and areas for these holes is being 
stored as reference data in a second set of in- 
formation. 

The learn mode sequence for the square- 
backed vehicle body 63 is simitar, but different in 
several respects and may be described as follows. 
Step 21: Wait for the square-backed vehicle body 



63 to move to rest in position within area 260. Step 
22: Snap an image of body 63 using camera 111. 
Step 23: Display the acquired image on CRT 118. 
Step 24: Enter interactive mode, so that operator 

5 can specify with the cross-hair cursor 296 the two 
window openings Q and P in the general manner 
described above with respect to the post holes in 
Rgure 8. When a bounded feature is recognized 
by the system 101, a suitable graphical symbol 

70 such as circle 298, shown in Figure 6, is drawn, 
about the feature. Step 25: Record the four average 
X, Y coordinates (X R ; X T ; Y s : Yu) for window open- 
ing Q and the four average X, Y coordinates (X R '; 
X T '; Y s '; Yy') for the window opening P, as part of 

15 the reference data for a third set of information 
stored in the software library. Step 26: Allow the 
human operator to type in (via keyboard 119) the 
English language description or the body style 63. 
Step 27: Derive the areas Aq and A p of the window 

20 openings Q and P, and record them as part of the 
reference data in the third set of information. Step 
28: Derive the distances (Drt; D S u; Dr't'; D s 'u ) 
for the length and width of the windows Q and P, 
and record these distances as part of the reference 

25 data for the third set of information. 



(2) Operating Mode For Vehicle Body Identification 

30 

The sequence of fully automatic steps per- 
formed by system 101 to identify the vehicle bod- 
ies as they are conveyed through the paint line 30 
by conveyor system 54 will now be described. The 

35 references made to buffers 1, 2, 3 and 4 refer to 
the four 256 x 246 frame buffers located within the 
processing buffer block 178 shown in Figure 2. 
Step 1: Turn on coprocessors 176, 180 and 182 to 
assist host computer 184, so that the identification 

40 of vehicle body style may be completed within 100 
milliseconds. Step 2: If Retry Count (explained 
later) equals zero, wait until a vehicle body is in 
position within rectangle 260. (The programmable 
controller 103 sends the vision system 101 a signal 

45 just before or just after a vehicle is in position in 
section 66 of paint line 30.) Step 3: Snap an image 
using camera 111 and put it in buffer 1. Step 4: 
Display the snapped image on CRT 118. Step 5: 
Acquire RGS pixel data for all rows of the buffer 1 

so in the general vicinity of the truck body, and put 
such results in the buffer 2. StejD 6: From the 
software library, get all of the reference data from 
the first set of information, which corresponds to 
the typical seven-foot pick-up truck body 61 . 

55 Step 7 is an iterative step, which is performed 
four times in order to detect the holes A, B, C, and 
D of truck bed 230. if indeed it is truck body 61 
which is present The substeps of Step 7 may be 
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described as follows. Substep (a): Obtain the cen- 
ter X, Y coordinates for the typical first hole from 
the reference data. Substep (b): For the rows X - 2 
to X + 2, move the actual RGS pixel data to buffer 
3. Substep (c): See if there Is a valley at pixel 
coordinates at Y - 3 to Y + 3. (By "valley" is 
meant a significant dip in the- RGS values, e.g., 
more than twenty units, which lasts for a minimum 
of two contiguous pixels and a maximum of four 
contiguous pixels, and then returns to a higher 
RGS level.) Substep (d): Note whether a valley was 
found. Substep (e): Move the next row from buffer 
to buffer 3, and repeat substeps (a) through (e) 
until all horizontal rows for the first hole have been 
examined. Substep (f): repeat the substeps (a) 
through (e) for the second through fourth holes. 
Substep (g): Determine if the valleys found in adja- 
cent rows of each hole are connected. Substep (h): 
If the valleys are connected, compute the areas 
and centroids of each of the four holes A-D. 

Step 8: See if the actual centroids of holes A-D 
computed in Step (7)(h) are within a distance of 
three pixels from the centroid locations on the 
holes A-D of the prototype body 61; if not, go to 
Step 21. Step 9: Compute actual distances D A b. 
Dbc. D C d and D A o and then check to determine if 
these actual distances are within a predetermined 
tolerance (e.g. three counts or pixels) of the values 
for these distances in the reference data from the 
first set of prototype information; if they are not, go 
to Step 21. Step 10: See if the hole areas A A , A B , 
A c and Ao are within a predetermined tolerance 
(e.g. plus or minus three pixels); if not, go to Step 
21 . Step 1 1 : Identify the vehicle body whose image 
was snapped in Step 3 as a seven-foot pick-up 
truck body since Steps 8, 9 and 10 are satisfied. 
Step 12: Write the English legend "seven-foot pick- 
up" on the CRT 118 as an overlay upon the image 
displayed in Step 4, and as an overlay on CRT 
118, draw white circles around the holes A-D to 
show that they have been recognized. Step 14: 
Send digital signal 1 to the programmable control- 
ler 103 over cable 126. Step 15: Go to. Step 2, and 
wait for another vehicle body to move into position. 
(Note that if the Steps 8 through 10 mentioned 
directly above were not satisfied, the vehicle body 
whose image was snapped in Step 3 may be: a 
six-foot pick-up truck, a square-back truck, or an 
unrecognizable vehicle or object, and the sequence 
of steps continues as described below.) 

In the preferred embodiment of the present 
invention, the software sequence is prepared so 
that the system 101 next checks to see if the yet- 
unidentified vehicle body In rectangle 260 is a six- 
foot pick-up truck, and if that is not satisfied, then 
checks to see if it is a square-back truck body. The 
sequence of Steps 21 through 30 used by system 
101 to determine whether it is a six-foot pick-up 



truck body is substantially identical to the se- 
quence of Steps 6 through 15 discussed above for 
the seven-foot pick-up truck body 61, and differs 
only in that: (1) the X, Y coordinate information for 

5 the holes and other reference data comes from the 
second set of reference information stored in the 
software library; (2) if body 62 is identified, a digital 
signal 2 is sent over cable 127 to programmable 
controller 103; and (3) if the checks (like those in 

10 Steps 9 and 10 above) are not satisfied, thus 
indicating that the body style is not recognized as 
a body 62, then go to Step 41 . 

The next sequence steps, Steps 41 through 52, 
are performed if the vehicle body is still unidenti- 

75 fied in order to determine if it is a square-back 
truck body. Step 41: From the software library, get 
the reference data from the third set of reference 
information, which corresponds to the prototype 
square-back truck. Step 42: Examine the relevant 

20 portion of the vehicle body image in buffer begin- 
ning five pixels to the right of the expected location 
of front edge R through five pixels to the left of the 
expected location of rear edge T, on a row-by-row 
basis, and record the actual X coordinates for the 

25 detected edges. Step 43: From the buffer 1. ac- 
quire RGS data for the columns in which edges S, 
U. S and u' are expected to occur and place 
these columns in the buffer 2. Step 44: Beginning 
with five pixels above the edge S and through five 

30 pixels below the edge U, examine RGS data in the 
columns and determine the actual Y coordinates of 
the horizontal edges S and U in the snapped 
image; do the same to locate the actual Y coordi- 
nates of the horizontal edges S' and u' of window 

35 opening P. Step 45: Compute the distances D RT , 
D S u. DrY, and D S V- Step 46: Compute the areas 
Aq and A p of the window openings Q and P. Step 
47: See if the distances computed in Step 45 are 
within a predetermined tolerance (e.g., three 

40 counts) of the corresponding values from the proto- 
type truck body recorded in the third set of in- 
formation; if not. go to Step 61. Step 48: Determine' 
if the areas Aq and Ap are each within a predeter- 
mined tolerance (e.g„ plus or minus 12 pixels) of 

45 the areas computed for the prototype vehicle body 
63; if not, go to Step 61. Step 49: Identify the 
vehicle body whose image was snapped in Step 3 
as a "square-back truck." Step 50: Overlay the 
descriptive field "square-back truck" on the CRT 

so 118. Step 51: Send a digital signal 3 via cable 128 
to the programmable controller 103. Step 52: Go to 
Step 2. 

If vision system 101 failed in its first pass 
through Steps 1 through 48 to successfully identify 
55 the vehicle body in area 66 whose image was 
snapped by camera 1 1 1 as one of the three body 
styles it has learned, the system 101 will retry the 
sequence of steps after performing a feature en- 
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hancement operation upon the snapped image in 
an effort to more clearly see the distinguishing 
features of the vehicle body in question, if it is still 
unable to identify the vehicle body, then the sys- 
tem 101 will so indicate by providing a message on 
CRT 118 and sending an appropriate signal the 
controller 103. The system 101 will only enhance 
the snapped image once. A software flag named 
"Retry Count" is used to determine when the sys- 
tem is trying a second time to identify the vehicle 
body from the image snapped in Step 3. The Steps 
61-67 described below are the preferred implemen- 
tation for the retry sequence just described, step 
61: If- Retry Count equals 1, then go to Step 65. 
Step 62: Set Retry Count equal to 1. Step 63: 
Execute a feature enhancement function called a 
"minimum filter" whose software call is "lLMED" in 
the IRI image processing library. This function is 
performed upon all pixels in the entire image 
stored in buffer 1, and the results of the enhance- 
ment are put back in buffer 1. In the preferred 
embodiment, this minimum filter function operates 
on a three-by-three pixel neighborhood, and it re- 
places the gray level of the center pixel by the 
smallest of the gray levels in its neighborhood. 
(This function has the effect of slightly enlarging 
and more sharply defining ail of the darkened 
areas, e.g. the post holes and window openings, of 
the image being analyzed.) Step_ 64: Go to Step 5 
(for the purpose of repeating all of the steps there- 
after, as needed, in an effort to identify the vehicle 
body in the snapped image from Step 3 and en- 
hanced by Step 63). Step 65: Put the message 
"unable to identify vehicle body" on the CRT 118. 
(This message is appropriate since the only way to 
get to Step 55 is from Step 61 . Step 66: Output a 
digital signal 4 to the controller 103 via cable 129. 
Step 67: Go to Step 2 (to wait for another vehicle 
body to identify). This completes the description of 
the preferred embodiment of the steps used to 
identify the vehicle body types presented at sec- 
tion 66 of the paint line 30. 



Intrusion-Sensing At Paint Booth Entrance 

Figure 11 is a simplified side elevational view 
taken along line 11-11 of Figure 1. The camera 113 
is used to observe a narrow area generally boun- 
ded by dashed lines 310 in Figure 1 and 311 in 
Figure 11, extending between the lens of camera 
113 and a diffused light panel 312 which contains a 
plurality of vertically arranged fluorescent tubes 
mounted in front curved reflectors and behind a 
translucent diffuser. Accordingly, the light panel 
312 appears as a reasonably bright and uniform, 
all-white surface. Accordingly, anything interposed 
between the light panel 312 in the field of view of 



camera 113 appears as a relatively dark object 
seen in profile by camera 113. In Figure 11, the top 
314 of the passageway 316 through the silhouette 
tunnel 68 is indicated by dashed lines. The front 

s end 318 of a typical vehicle body 320 is shown 
protruding from the exit 322 of the silhouette tunnel 
68. The body 320 could be any one of the bodies 
61, 62 or 63 since the shape and size of each of 
their front ends is very similar, especially when 

to viewed in profile. When the vision system 102 to 
which the camera 113 is connected detects the 
front end 318 of vehicle body 320 reaching a 
predetermined location such as is indicated by the 
vertical line 324, the vision system 102 will send a 

75 signal via interface board 142 and cable 146 to the 
programmable controller 104, which can be used 
as a zone start signal. 

Figure 12 shows a human being 330 coming 
out of the exit 322 of the silhouette tunnel 68. His 

20 profile is readily visible against the lighting panel 
312 by camera 113. The vision system 102 can 
readily distinguish between a typical vehicle 320 
and the typical human being 330 by determining 
the average pictorial characteristics of each. Spe- 

25 cifically, I have determined that the front end 318 
of the average truck 320 exiting from passageway 
316 of silhouette tunnel 68 has a centroid 322 
which is much lower (that is, vertically displaced 
below) the average centroid 334 of a human being 

30 330 walking in an erect or semi-erect position as 
he exits from the passageway 316. 



(1) Learn Mode for Intrusion-Sensing 

35 

The preferred sequence of steps used in a 
learn mode to teach the vision system 102 what 
the average centroid of the front end 318 of vehicle 
body 320 is and what the average centroid 334 of 

40 typical human 330 will now be described. Step 1 : 
Turn on the coprocessors of vision system 102. 
Step 2: Snap an image of the empty background 
(that is, when no car body or human body is 
present) as a reference frame and store it in buffer 

45 2. Step 3 is an iterative step repeated twenty times 
and consists of the following substeps. Substep (a): 
With a human being walking from right to left, snap 
an image when the human being gets very near 
the exit 322 of the passageway 316 and store it in 

so buffer 1 . Substep (b): Subtract buffer 2 from buffer 
1 to obtain a blob, and store the blob temporarily in 
buffer 3. Substep (c): Derive the centroid of the 
blob in buffer 3. Substep (d): Add the centroid of 
the blob in buffer 3 to a variable (which is initially 

55 zero) named "centroid total." Substep (e): While a 
human being walks from left to right in Figure 11, 
and is just exiting the passageway 316, snap an- 
other image and store it in buffer 1. Substep (f): 
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Buffer 2 is subtracted from buffer 1 and the results 
stored in buffer 3. Substep (g): The centroid of the 
blob in buffer 3 is computed and added to the 
current value of "centroid total." Substep (h): Sub- 
steps (a) through (g) are repeated 19 times. Step 4: 5 
Calculate a human centroid average by dividing the 
value of centroid total by 40. Step 5: Record the 
human centroid average in the library as reference 
data. Step 6: Perform the following substeps (a) 
through (c) a predetermined number of times, such io 
as twenty, for each of the three different body 
styles to be recognized. Substep (a): Snap an 
image using camera 113 of a car body as it begins 
to exit as shown in Figure 1 1 from the passageway 
316 of the silhouette tunnel 68, and store the 75 
snapped image in buffer 1. Substep (b): Subtract 
buffer 2 from buffer 1 , and put the results in buffer 
3. Substep (c): Compute the centroid of the blob in 
buffer 3, and add it to the variable centroid total 
(which is initially zero at the start of Step 6). Step 20 
7: Compute the vehicle centroid average by divid- 
ing the value of centroid total by 60. Step 8: The 
vehicle centroid average is recorded as reference 
data in the software library. This completes the 
learn mode for the intrusion and zone start signal. 25 



(2) Operating Mode for Intrusion-Sensing 

During paint line operation, the following se- 30 
quence is presently preferred for producing the 
intrusion and zone start signals. Step 1: Turn on 
the coprocessors. Step 2: Snap an image of an 
empty scene (i.e., no vehicle body, human or for- 
eign object present) and store it in buffer 2. Step 3: 35 
Read the average human centroid value and the 
average vehicle body centroid value -from the soft- 
ware library. Step 4: Snap an image with camera 
111, and store it in buffer 1. Step 5: Subtract buffer 
1 from buffer 2 and store the results in buffer 3. 40 
Step 6: If buffer 3 is empty (indicating that nothing 
has changed in the field of view of camera 113), 
then go to Step 4. Step 7: Compute the centroid of 
the blob in buffer 3. Step 8: If the Y coordinate of 
centroid of the blob is equal to the vehicle centroid 45 
average within a tolerance of two counts, and the X 
coordinate of the centroid of the blob has reached 
a predetermined point then issue a start zone 
signal (digital signal 1) over cable 147 and go to 
Step 11. Step 9: If the Y coordinate of the centroid so 
of the blob in buffer 3 is equal to the average ' 
human centroid value within a predetermined toler- 
ance (such as two counts), then issue a signal 
(digital signal 2) on cable 148, indicating human 
intrusion at the entrance to the electrostatic paint 55 
spray booth 36, and go to Step 1. Step 10: If this 
step is reached, the object in field of view of 
camera 113 has not been recognized as a vehicle 



body or a human body. Accordingly, issue digital 
signal 3 over cable 149. indicating that an unrecog- 
nizable object has intruded into the field of view of 
camera 113. 

To determine when the unrecognized object 
has moved out of the field of view yet. Steps 1 1 
through 13 are performed. Step 11: Snap another 
image with camera 113 and store it in buffer 1. 
Step 12: If buffer 1 is equal to buffer 2, go to Step 
4. Step 13: go to Step 11. 



E Monitoring Clearance Between Roof Machine & 
Vehicle Body 

Figures 13 and 14 are a side elevationaf view 
and an end view respectively taken along line 13- 
13 and line 14-14 of Figure 1. Figures 13 and 14 
illustrate in simplified form a ceiling-mounted roof 
machine 81 including platform 80 poised above the 
front hood section 248 of vehicle body 62. The 
platform 80 provides the mounting support for the 
three paint bells 77-79. The platform 80 is in turn 
supported by a ceiling-mounted superstructure 
340, which is shown in highly simplified form since 
the particular design of the superstructure 340 and 
platform 80 form no part of the present invention. 
The platform 80 is movable in a vertical direction 
under the control of hydraulic cylinder 342 whose 
rod 344 has sufficient downward travel to allow the 
bells 77-79 to paint the inside bottom surface of 
the beds 230 and 240 of truck bodies 61 and 62 
and sufficient upward travel to provide the needed 
clearance above the cab sections 236 and 246 of 
truck bodies 61 and 62. The superstructure 340 
may include slotted guide bars 346-349 rigidly 
fastened to transverse crossbeams 350 and 352. 
while the hydraulic cylinder 342 is connected to a 
longitudinal beam 354 between crossbeams 350 
and 352. A pair of guide frames 356 and 358 each 
have four roller guide means, such as captive roller 
assemblies 359. to siidably interconnect the guide 
frames to guide bars 346-349, to stabilize the plat- 
form 80 and prevent it from moving in any direction 
except in the vertical direction allowed by the slots 
and members 346-349. 

The platform 80 is shown in a simplified fash- 
ion in Figures 13 and 14. Typically, it includes 
several dozen valves and hoses for paints of var- 
ious colors, solvents, air lines, and the like (not 
shown), familiar to those in the painting equipment 
art Attached to the righthand side of platform 80 
by a rigid bracket 360 is a rectangular high-con- 
trast optical target 362 which includes a bright 
white central area 364 surrounded by a broad black 
rectangular stripe 366. The target 362 provides an 
easily-recognized, highly visible optica! feature for 
the camera 114 shown in Figure 1 to focus upon. 
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The bracket 360 includes a transverse horizontal 
section 368 and a vertical extension 370 extending 
downwardly therefrom, as best shown in Figure 14. 
If desired, the bracket 362 may be omitted entirely, 
by mounting the target 362 directly upon the end 
of the platform 80. However, the bracket 362 per- 
mits the target to hang outside of the bed 240 of 
the vehicle body during the painting of the bed by 
the roof machine 81 as illustrated in Figure 15. 

As best seen in Figure 1, the third camera 114, 
which is the second camera of vision system 102, 
has a field of view indicated by dashed lines 380 
which encompasses the lower portion of roof ma- 
chines 81 and a portion of the vehicle body there- 
under (if any). As in Figures 11 and 12, diffused 
white lighting panels 382 are vertically along the far 
wail opposite the camera 114 so that the lower 
postion of the roof machine, and the car body will 
appear as relatively dark objects in profile against 
the white background provided by lighting panels 
382. Like the panel 312, each lighting panel 382 
typically includes a plurality of fluorescent tubes, 
one or more curved reflectors therebehind, and a 
translucent white panel on the front side to provide 
a uniform white background. For convenient refer- 
ence, the outline of the panels 382 is shown in 
Figure 13 by dashed lines. 

Using camera 114, the vision system 102 mon- 
itors the clearance between the roof machine 81 
and, in particular, the bells 77-79 and the vehicle 
bodies to be painted, such as vehicle body 62. The 
bottom of paint bells 77-79 are all at the same 
elevation as shown in Figure 14; therefore, the 
camera 114 generally only sees the profile of the 
bell 77. While the clearance between the paint 
bells 77-79 and the truck bodies can vary from a 
minimum of about six inches (0.15 meter) to about 
two feet (0.6 meter), the closer distances, such as 
seven to nine inches (0.18 to 0.23 meter), are 
preferred for electrostatic paint spray machines be- 
cause paint transfer efficiency improves with such 
closer distances. In an electrostatic paint spray" 
operation, the part to be painted, such as a metallic 
vehicle body, is typically held at about 0 volts or 
ground potential, and the paint is highly negatively 
charged to as much as 70,000 volts or so. Accord- 
ingly, the paint is literally drawn by electric forces 
to the suface of the object to be painted resulting 
in relatively high transfer efficiency with a minimum 
of overspray and paint mist. Because the paint 
travels so quickly from the bells to the truck body, 
the light from the diffuse lighting panel 382 is not 
detrimentally obscured by the paint spray, thus 
permitting the system of my invention to measure 
the approximate clearance (i.e. the distance) be- 
tween the paint bells 77-79 and nearby portions of 
the truck body. 

As is well known in the art, it is important to 



maintain the desired clearance between the paint 
bells and the object being painted in order to 
properly distribute the paint over the object and to 
avoid the possibility of fire due to arcing between 

5 the bells and body being painted. Typically, the 
paints being applied are rather flammable, and thus 
the possibility of fire due to arcing must be care- 
fully guarded against. In the prototype system of 
my invention, an area of interest 390, which may 

w be conveniently called a bubble, is defined beneath 
the bell 77 and extends downwardly in the vicinity 
of the nearby portion 248 of the truck body 62. The 
size of the bubble is variable, since the bubble in 
fact does not physically exist; it is a construct of 

75 the software of the vision system 102 which moni- 
tors the region of interest 390 that exists between 
the bells 77 and truck body. The size and shape of 
the area of interest may be arbitrarily defined within 
limits. For convenience, the prototype system of 

20 my invention considers the area of interest to be 
circular as shown in Figures 13 and 15. If any 
portion of the vehicle body invades the area of the 
bubble, the vision system 102 so notifies the prog- 
rammable controller 104 so that appropriate action 

25 may be taken, such as the immediately raising 
platform 80. The vision system 102 determines the 
precise location of the relevant portion of the ve- 
hicle body within the field of view of camera 114 
and also determines the precise location of the 

30 target 362. Knowing the location of the target, and 
the physical relation between the position of the 
target and the bell 77, it Is a simple matter for the 
vision system 102 to calculate the location of the 
bubble. For example, if the centroid of the target 

35 area 364 is at Xj.iYk, the system 102 can calculate 
the center 394 of the bubble 390 and the circum- 
ference of the bubble at coordinates X Ll Y M by 
knowing the values of the horizontal and vertical 
dimensions indicated by arrows 396 and 398. The 

40 benefit of using the target 362 is that it is highly 
visible and readily found, even as lighting con- 
. ditions change somewhat. To help make the target 
more visible, a lighting panel (or two) may be 
located somewhat behind and to the side of the 

45 camera, 11 4, if desired. Those in the art should 
appreciate that, if desired, the vision system 102 
could be programmed to recognize the shapes of 
the bells 77 or distinctive features of the platform 
80 located nearby the bell (if any). While this is 

so possible, I have found it much better to use a high 
contrast target, such as target 362, which has a 
predetermined fixed relationship with respect to the 
portions of the platform 80, namely bells 77-79, of 
interest in terms of maintaining a specified clear- 

55 ance. 

Figure 15 is a side elevational view showing 
the platform 80 lowered by hydraulic cylinder 342 
into the bed 240 of truck body 62 in order to paint 
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the interior bottom surface 400 of the bed. As 
previously mentioned, since the support bracket 
340 upon which the target 362 is mounted enables 
the target to hang outside of the truck bed 240, the 
clearance between the paint bells 77-79 and the 
inside bottom 400 can be determined, even though 
the bells 77-79 and part of the lower portion of 
platform 80 are hidden by sidewall 241 of the bed 
240. 

If the target 362 is mounted directly to the 
platform 80 in about the same location shown in 
Figures 13 and 15, the target would partially dis- 
appear within the bed 240 when the roof machine 
81 is in the position shown in Figure 15. This could 
present a problem in terms of properly identifying 
the location of the bells 77 using the target 362. To 
avoid this problem, a second target 402, preferably 
of somewhat different geometric shape (perhaps 
square instead of rectangular) could be provided. 
Then, when the vision system recognized that the 
first target 362 was about to be obscured by the 
sidewall of the bed 240, it could look to target 402 
and calculate therefrom the position of the bell 77 
relative to the bed 240. 

In prototypes of the present invention, the verti- 
cal movement of the platform 80 is controlled by 
programmable controller 104 in synchronization 
with the position of the truck body being painted. 
This synchronization is determined by the control- 
ler 104 keeping precise track of the incremental 
travel of the belt conveyor 56, and hence the truck 
bodies thereon, due to frictionai (or other) engage- 
ment of the pallets thereunder with the belt con- 
veyor. The system of the present invention could 
be programmed, however, to provide control sig- 
nals to specify to the controller 103 how much to 
raise or lower the platform 80 in order to ensure 
proper clearance between the paint bells 77-79 and 
the truck bodies, as will be further explained short- 
ly. 

Figures 16 and 17 are front views of a CRT, 
such as CRT 138 or optional CRT 153, with each 
CRT displaying the image seen in the field of view 
of camera 114. Figure 16 shows the roof machine 
80 and paint bells 77 positioned above, but too 
close to, the roof of the cab section 246. so that a 
portion 410 of the cab section 246 has entered the 
guarded zone or bubble 390, which is often re- 
ferred to as a "broken bubble" condition. This is 
appropriately communicated to observers of the 
CRT by placing or repetitively flashing a suitable 
legend 418, such as "Warning - Broken Bubble," 
as an overlay on the image being displayed on the 
CRT and/or by deleting that portion of the bubble 
390 in contact with the nearby portion 410 of the 
object being painted, as shown in Figure 16. Alter- 
natively, any other visual or aural indication may be 
used to call attention to the broken bubble con- 



dition. 

Figure 17 shows yet another simplified view of 
the screen of CRT 138 (or 153) having not one. but 
four, areas of interest, three of which are indicated 

5 by concentric bubbles 420, 422 and 424 centered 
beneath the bells 77 and of three different diam- 
eters. The fourth zone of interest is bubble 426 
which is located diagonally off the lower lefthand 
comer 428 of platform 80. The three concentric 

io bubbles 420-424 of Figure 17 illustrate that the 
system 102 can, if desired, calculate areas of inter- 
est of different sizes, including overlapping and/or 
concentric areas. More importantly, Figure 17 illus- 
trates one method of controlling the clearance or 

75 spacing between the bells 77 and the nearby por- 
tions of the vehicle body to be painted. Concentric 
bubbles 422 and 424 may be respectively used to 
define the minimum desired clearance and the 
maximum desired clearance between the bells 77 

20 and the object to be painted. Thus, the vision 
system 102 can command the platform 80 of the 
roof machine to be lowered until the bubble 424 is 
broken by the object to be painted. However, the 
bubble 422 should not be broken by the object to 

25 be painted, and if it is, the roof machine should be 
raised slightly so that it is no longer broken. The 
inner concentric bubble 420 can be used as the 
safety bubble which, if broken by the object to be 
painted, causes the platform 80 to be pulled up 

30 rapidly and an appropriate warning given of this 
abnormal condition (which should not have oc- 
curred). 

The fourth bubble 426 illustrates that the vision 
system 102 can readily define areas of interest 

35 other than those directly beneath or near the paint 
bells 77-79. Thus, if the corner 428 of platform 80 
comes too close to the truck body, so that the 
bubble 426 is broken, the vision system 102 can 
notify the controller 103 so that appropriate action 

40 is taken. The location of bubble 426 can be readily 
calculated once the centroid of the target area 364 
is determined, using the same basic techniques 
used to calculate center 394 of bubble 390. Having 
described the general operation of the vision 102 

45 with respect to the images taken by camera 1 1 4, a 
more detailed description of a preferred sequence 
of operation with respect to the clearance between 
bells 77-79 and the truck bodies will now be de- 
scribed. 

50 

(1) Set-Up Procedures For Clearance Monitoring 

In order to set up camera 114 to properly 
55 perform in the prototype system of the present 
invention, I first set the dual high-low thresholds 
used for locating the target 362 and the single 
threshold used to binarize the image when the 
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profile of the vehicle body and the roof machine is 
to be determined. The dual threshold levels are 
used to enhance the image from which the location 
of the target will be determined, and they are set in 
the following manner. First, the RGS level at which 
the dark background objects begin to bleed into 
the black perimeter 362 of the target 366 is deter- 
mined, and the low threshold is set at a predeter- 
mined amount (such as five RGS levels) there- 
below. Next, the RGS level at which the white 
portion 364 of the target begins to bleed into the 
black perimeter 366 of the target 362 is deter- 
mined, and the high threshold is set a predeter- 
mined amount (such as five levels) therebelow. 
Thereafter, an image of the target in a known 
location is snapped, and the area in X coordinates 
of the centroid of the target is computed and 
recorded as. reference data in the software library. 
Alternatively, the white area 364 of the target may 
be hard-coded as a constant in the applications 
software, if desired. 



(2) Operating Mode For Clearance Monitoring 

The sequence of operating steps may now be 
described. Step 1: Vision system 102 using camera 
114 snaps an image of the portions of the roof 
machine 81 and vehicle body within the field of 
view of camera 114, and the snapped image is 
loaded into buffer 1 . Ste£ 2: A vertical strip from 
the snapped image in buffer 1 that is centered 
about the vertical center line Xj of the target 362 
and is slightly wider than the target loaded into 
buffer 2 using an "enhance interval" image pro- 
cessing function which reduces gray scale values 
beneath the minimum threshold level to zero and 
raises gray scale values above the maximum 
threshold value to a gray scale value of 255. The 
result of this process is that one or more blobs are 
present in buffer 2, one of which is the target. Step 
3: Perform blob analysis to locate all blobs within 
buffer 2 by performing three substeps as follows. 
Substep (a): Locate ail edges or contours which 
might be associated with blobs by processing buff- 
er 2 with a "run-length encoding" image process- 
ing algorithm (such as "IRLEN" from the IRI soft- 
ware library), and load the result back into buffer 2. 
Substep (b): Identify all discrete, non-connected 
blobs by processing buffer 2 with a blob labeling 
algorithm (such as "LABELPIC" from the IRI soft- 
ware library). Substep (c): Compute the area and 
centroid of each discrete biob identified in substeb 
(b). Step 4: For each discrete, non-connected blob 
identified by Step 3, compare the actual area and 
X coordinate of the centroid for that blob against 
reference data stored during the set-up prepara- 
tions for the area and centroid of target 364. If the 



area for a blob is within a first tolerance (such as 
plus or minus three pixels), and if the X coordinate 
is within a second tolerance (such as plus or minus 
two pixels of the X coordinate from the reference 

5 data for the target), then identify the blob in ques- 
tion as the target Step 5: Compute the X, Y 
coordinates of the center 394 of the bubble 390, 
which represents the area of interest to be mon- 
itored for invasion by the vehicle body. Step 6: 

70 Having identified the enter 394 of the bubble 390, 
generate an all-white bubble 390 and put the bub- 
ble into buffer 3, which previously was filled with 
zeros (so that it would initially be interpreted as 
being all black). Also, erase buffer number 2 (fill it 

75 with zeros, and also put the bubble in buffer 2). 
Step 7: Locate the car body profile and roof ma- 
chine profile in the snapped image of buffer 1 by 
binarizing the image, such that every object within 
the snapped image, except for the bright white 

20 background of lighting panels 382 appears black, 
and put the results in buffer 4. Step 8: Map buffer 4 
into buffer 3 by having the black values (level zero) 
overwrite the white values (level 255) in buffer 3. 
(Note that if the vehicle body intrudes into the ali- 
as white bubble, this can be detected by examining 
buffer 3 to determine if any white pixels associated 
with the bubble have turned black, which is done in 
Step 9.) Step 9: Perform a logical AND operation 
by mapping buffer 3 into buffer 2 so that only 

30 pixels which are allwhite (level 255) in both buffer 2 
and buffer 3 remain in buffer 2, with all other pixels 
being given a zero value. Then, compute the area 
of the bubble (by counting pixels) in buffer 2, and 
compare the pixel count against the area (i.e., pixel 

35 count) of the original size of the bubble. If the 
number of pixels in the bubble of buffer 2 is more 
than a predetermined number of pixels less than 
the original bubble (such as ten pixels), system 102 
informs programmable controller 104 that a bubble 

40 has burst. At this point, the vision system 102 
stops (i.e., does not further process images 
snapped by camera 114) and displays the actual 
burst bubble condition which has been detected 
(as in Figure 16). Step 11: If the bubble had not 

45 burst, go to Step 2 to repeat the process. 

The foregoing sequence of Steps 1 through 10 
is performed by the vision system 102 in about 80 
milliseconds. Thus, it is possible for the vision 
system to perform this sequence ten times (or 

so more) per second, and thereby effectively continu- 
ously monitor the clearance between the bell 77 
and the vehicle body thereunder. In order to permit 
the operator/observer of the CRT 138 (or 154) to 
verify that the system 102 is performing correctly, 

55 the vision system 102 displays only a small fraction 
of the total number of Images of the scene 
snapped by camera 114. Specifically, I have found 
that by displaying only about one out of ten images 
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of the type generally shown in Figures 16 and 17, 
humans watching the CRT are able to visually 
verify that the bubble 390 appears to be properly 
placed between the paint bell 77 and the vehicle 
body. This helps convince them that in fact that the 5 
vision system 102 is working properly. 

A technique I found helpful to achieving the 
high speed operation just described is processing 
only the regions of interest where the objects to be 
detected are sure to be located. Thus, in Step 2. w 
for example, my software is programmed only to 
look at the vertical strip in which the target 362 
must be located. 

With respect to the clearance monitoring op- 
eration performed by camera 114 and vision sys- ;s 
tern 102, there is a need to test the system under 
actual operating conditions without actually making 
the paint bells 77-79 come too close to the truck 
body being painted, in the prototype system of the 
present invention, non-destructive testing of the 20 
system 102 is accomplished by simply enlarging 
the diameter of the bubble so that during a portion 
of the painting operation of a car body, the car 
body intrudes upon the bubble, thus causing sys- 
tem 102 to notify the programmable controller 104. 25 



F. Epilogue 

While the foregoing detailed description has 30 
related to the use of vision systems on automotive 
vehicle paint lines, particularly those with electro- 
static paint spray booths, those skilled in the art wil 
appreciate that the vision system and methods of 
the present invention are equally applicable to oth- 3S 
er automated painting equipment, including paint- 
ing equipment with non-contact reciprocating ap- 
plicators, spraying wands and the like. In other 
applications, where the amount of paint splatter or - 
overspray is greater, it is only necessary to ensure 40 
that the camera lens and light sources remain 
relatively clean, so that the vision system will still 
be able to see the objects of interest within its field 
of view. Since the cameras can be and normally 
are mounted a significant distance from the scene 45 
they are observing, this is usually not a problem. 

As used herein and in the claims below, the 
following selected terms may be further defined as 
follows: The term "pictorial attributes" is meant in a 
general sense to include any visually perceptible or 50 
visually distinguishable attributes of an object, in- 
cluding a person, within the field of view of a 
camera means. The term "camera means" in- 
cludes standard RS170 output video cameras, as 
well as any other solid-state or tube-type camera 55 
device (e.g., CCD array, vidicon) which produces 
electronically processable signals representing a 
two or three-dimensional scene, by perceiving or 



discerning images of the scene using non-contact 
optical techniques. The term "automotive vehicle 
bodies" includes car bodies, truck bodies or both 
types of bodies, or major portions thereof, such as 
by way of example and not limitation hoods, fend- 
ers, passenger compartments and truck beds. The 
term "image analysis system" includes any type of 
digital or analog processing system for extracting, 
distinguishing, enhancing, or otherwise analyzing 
electronic or other signals representing information 
corresponding to all or a two (or more) dimensional 
portion of an image perceived or discerned by a 
camera means. 

The foregoing detailed description shows that 
the preferred embodiments of the present invention 
are weil-suited to fulfill the aforementioned objec- 
tives and provide the advantages above-stated. It is 
recognized that those in the art may make various 
modifications or additions to the preferred embodi- 
ments chosen to illustrate the present invention 
without departing from the spirit and scope of the 
present invention, which is defined by the appen- 
ded claims, including all fair equivalents thereof. 



Claims 

1. Vision apparatus (101; 102) for automated 
painting equipment (30; 103; 104), comprising: 

(a) at least one camera means (111: 113; 
114) for acquiring successive images of a first 
scene (260; 310, 311; 382) from a first portion (66; 
68, 312; 36, 80. 382) of the painting equipment 
including at least a first movable object (61 ; 62; 63: 
330) to be identified via selected pictorial attributes 
(A - D; A - D; Q - P; 318; 332. 330. 334; 240. 246. 
248) thereof; and 

(b) an image analysis system (117: 137; 170) 
for digitally processing successive images received 
from said camera means, said system including 
means (174-188) for recognizing selected pictorial 
attributes of the first object when such attributes 
are present in at least one of the successive im- 
ages, means (174-188) for comparing the recog- 
nized attributes of the first object with correspond- 
ing reference data from a first set of information 
previously stored in the system, and means (182- 
184) for identifying the first object as being present 
in the first scene when a match within a predeter- 
mined first tolerance exists between at least one of 
the recognized pictorial attributes and correspond- 
ing reference data from the first set of information. 

2. Vision apparatus as in claim 1, further com- 
prising: 

(c) means (122-130; 142-150. 184) for com- 
municating the identification of the first object as 
being present in the first scene to the automated 
painting equipment (30; 103; 104). 
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3. Vision apparatus as in claim 1 , further com- 
prising: 

(c) means (118, 120; 138; 140; 153) for visu- 
ally communicating the identification of the first 
object as being present in the first scene to a 
human being. 

4. Vision apparatus as in claim 3, wherein 
means (c) includes a two dimensional image dis- 
play means (118; 138; 153) for displaying at least 
part of the scene containing the first object for 
which a positive identification has been made. 

5. Vision apparatus as in anyone of claims 1 to 

4, wherein the image analysis system 
(1 17:137:170) further includes: 

means (184) for determining when the first object 
has moved into the scene sufficiently to have 
reached a predetermined location within the first 
scene. 

6. Vision apparatus as in anyone of claims 1 to 

5, wherein the image analysis system further in- 
cludes: 

means (122-130; 142-150, 184) for sending a signal 
to the painting equipment when the first object has 
reached the predetermined location within the first 
scene. 

7. Vision apparatus as in anyone of claims 1 to 

6, wherein the image analysis system (137;170) 
further includes: 

means (182, 184) for recognizing the entry of a 
human being into the first scene. 

8. Vision apparatus as in claim 7, wherein the 
first portion of the painting equipment is the an 
electrostatic paint spraying area (36), and the first 
scene (310. 311) thereof includes at least one 
entrance way (322) through which a human being 
might enter the painting spraying area. 

9. Vision apparatus as in anyone of claims 1 to 
8, wherein the first scene (382) includes a second 
movable object (77;80;362;402) to be identified via 
selected pictorial attributes (364.366) thereof, and 
wherein the image analysis system (137;170) fur- 
ther includes: 

means. (174-188) for recognizing selected pictorial 
attributes (Xj, Yk) of the second object (362) when 
such attributes are present in at least one of the 
successive images at the same time selected at- 
tributes (246) of the first object (62) are present 
therein, means (174-188) for comparing the recog- 
nized attributes of the second object with cor- 
responding reference data from a second set of 
information about the second object previously 
stored in the system, and means (182-184) for 
identifying the second object as being present in 
the first scene when a match within a predeter- 
mined second tolerance exists between at least 
one of the recognized attributes of the second 
object and the corresponding reference data from 
second set of information. 



10. Vision apparatus as in claim 9, wherein the 
image analysis system further includes: 

means (174-188) for determining from at least one 
of the successive images a distance between a 
s selected first portion (77; 428) of the second object 
(80) and a portion (242; 246; 248) of the first object 
(62) nearby the selected first portion of the second 
object 

11. Vision apparatus as in claim 10, compris- 

70 ing: 

(c) means (174-188) for comparing the deter- 
mined distance against a plurality of predetermined 
values representing maximum and minimum op- 
erating distances to be maintained between the 

75 selected first portion of the second object and the 
first object during painting, and 

(d) means (142-151. 184) for communicating 
to automated painting equipment whether the de- 
termined distance is less than, between, or greater 

20 than the predetermined distances. 

12. Vision apparatus, as in claim 10 or 11, 
further comprising: 

(c) means (142-151, 184) for communicating 
to the automated painting equipment that the deter- 

25 mined distance is less than a specified minimum 
distance. 

13. Vision apparatus as in claim 12, wherein 
the minimum distance is a minimum safe clearance 
distance, and wherein the apparatus further com- 

30 prises: 

(d) means (142-151, 184) for commanding 
the automated painting equipment to take remedial 
action when the determined distance is below the 
minimum safe distance so that automated painting 

35 equipment will cause the determined distance to 
become larger promptly. 

14. Vision apparatus as in anyone of claims 9 
to 13, further comprising: (c) means (138; 153) for 
visually displaying, as part of at least one of the 

40 images of the first scene, at least one graphical 
symbol (390; 420; 422; 426) representing a mon- 
itored area of interest adjacent to a selected first 
portion (77; 428) of the second object (80) into 
which the first object (62) might possibly intrude, 

45 but is not supposed to intrude. 

15. Vision apparatus as in claim 14, wherein 
the selected first portion of the second object in- 
cludes at least one non-contact paint spray applica- 
tor (77), and the first object (62) is an object to be 

so painted at least in part by the one paint spray 
applicator. 

16. Vision apparatus as in claim 15, wherein 
the automated paint equipment (30) is a paint line 
for painting automotive vehicle bodies (61; 62; 63), 

55 and the first object (240; 246; 248) is a at least part 
of an automotive vehicle body. 
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17. Vision apparatus as in claim16, wherein the 
camera means (114) is located to acquire succes- 
sive images of at least one upper side portion of 
the automotive vehicle body (62), and the second 
object is an automated paint spray machine (80, 
81) for painting the body from above the body. 

18. Vision apparatus as in anyone of claims 1 
to 17, further comprising: 

(c) means (174-188, 204) for selectively dis- 
playing, when the vision system is processing im- 
ages at a rate higher than the ability of a human 
being to follow closely, only a few of many images 
being successively received by the image analysis 
system, whereby the human being may visually 
verify an appearance of correct operation of the 
system. 

19. Vision apparatus as in claim 18, further 
includes: 

(d) means (138; 174-188, 204) for visually 
displaying, as part of the few images selectively 
displayed of the first scene, at least one graphical 
symbol (390; 420; 422; 426) representing a mon- 
itored area of interest adjacent to a selected first 
portion of the second object into which the first 
object might possibly intrude, with said displayed 
symbol being disposed between displayed images 
of the first object (62, 240, 246, 248) and the 
selected portion (77, 428) of the second object 
(80). with the displayed symbol being substantially 
stationary to with respect to the displayed image of 
the second object and contactable by the dis- 
played image of the first object when the first 
object gets too close to the second object 

20. A method of using a machine vision system 
(101; 102) in connection with the operation of auto- 
matic painting equipment (30; 103; 104), compris- 
ing the steps of: 

(a) acquiring, through the use of at least one 
camera means (111; 113; 114), successive images 
of a first scene (260; 310; 31 1 ; 382) associated with 
at least a first portion (66; 68, 312, 36, 80, 382) of 
the automated painting equipment the first scene 
from time to time including therein at least a first 
movable object (61; 62; 63; 330) having selected 
pictorial attributes (A - D; A' - D; Q - P; 318, 332; 
330, 334; 240. 246, 248); and 

(b) processing successive images received 
from the camera means using an image analysis 
system (117; 137; 170) by recognizing selected 
pictorial attributes of the first object within the first 
scene, comparing the recognized attributes of the 
first object with corresponding reference data from 
a first set of information previously stored in the 
image analysis system, and identifying the first 
object as being present in the scene when a match 
within a predetermined first tolerance exists be- 



tween at least one of the recognized attributes and 
the corresponding reference data from the first set 
of information. 

21. A method as in claim 20 including a task 
5 definition procedure performed before steps (a) 

and (b), further comprising the steps of: 

(c) acquiring an image of a typical first ob- 
ject (230; 240; 250) to be identified: 

(d) placing the image analysis system in a 
w learn mode; and 

(e) instructing (296) the image analysis sys- 
tem as to which of the pictorial attributes (A - D; A 
- D; Q, P) of the typical first object are to be 
stored as part of the first set of information for use 

75 during step (b). 

22. A method as in claim 20. further compris- 
ing the steps of: 

(c) acquiring at least one reference image of 
the first scene when the first scene does not con- 

20 tain the first object; 

(d) storing the reference image in the mem- 
ory (194; 196; 198) of the image analysis system; 

(e) comparing a subsequently acquired im- 
age of the first scene with the stored reference 

25 image to determine whether an object to be evalu- 
ated has moved into the scene. 

23. A method as in claim 22, further compris- 
ing the step of: 

(f) producing a signal which is sent to the 
30 automated painting equipment when an object (61; 

62; 63; 330) is detected in the first scene. 

24. A method as in claim 23, wherein the first 
object is inanimate, and wherein the signal in- 
dicates that the detected object has been identified 

35 as the first object. 

25. A method as in claim 23, wherein the signal 
indicates that a human being (330) has moved into 
the first scene. 

26. A method as in claim 23, wherein the signal 
40 indicates than an object has been detected as 

being present in the first scene, and that the image 
analysis system has been unable to identify it. 

27. A method as in claim 20, wherein the first 
object is an automotive vehicle body selected from 

45 a predetermined group of different styles of auto- 
motive vehicle bodies (61; 62; 63). 

28. A method as in anyone of claims 20 to 27, 
wherein the selected pictorial attributes (A-D) of the 
first object to be recognized are small features of 

so the first object which collectively occupy less than 
15% of the total area of the first object as viewed 
by the one camera means, and which occur at 
known locations (X A , Y A -X D , Y D ) of the first object 
(61). 

55 29. A method as in claim 28, wherein step (b) 
further comprises the substep of calculating the 
area of at least one of the small features. 
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30. A method as in claim 29, wherein step (b) 
further comprises the substep of determining the 
centroid of the area of the one small feature. 

31. A method as in claim 30, wherein step (b) 
further comprises the substep of comparing the 
area and centroid of the one small feature with 
expected values of the area and centroid for the 
one small feature previously stored in the system. 

32. A method as in anyone of claims 20 to 31 . 
wherein step (b) further comprises the substeps of: 
recognizing selected pictorial attributes (77; 362; 
402) of a movable second object (80, 81) within the 
first scene, comparing the recognized attributes of 
the second object with corresponding reference 
data from a second set of information previously 
stored in the image analysis system, and identify- 
ing the second object as being present in the first 
scene when a match within a second predeter- 
mined tolerance exists between at least one of the 
recognized attributes of the second object in the 
first scene and the corresponding reference data 
from the second set of information. 

33. A method as in claim 32, wherein step (b) 
further comprises the substeps of: 

determining the relative location of the second ob- 
ject within the first scene, and calculating the per- 
ceived clearance between a portion (240; 246; 248) 
of the first object (62) and a first portion (77; 426) 
of a second object (80), and wherein said first 
portion of the second object has a predetermined 
substantially constant relationship relative to the 
recognized attributes (362; 402) of the second ob- 
ject (80). 

34. A method as in claim 33, wherein the 
recognized attributes of the second object are as- 
sociated with a high contrast target (362) of pre- 
determined geometric shape. 

35. A method as in claim 34, wherein the 
second object (80) is at least part of a roof ma- 
chine (81), and the first portion of the second 
object includes at least one electrostatic paint 
spray bell (77). 

36. A method as in claim 35, wherein the first 
object (62) is an automotive vehicle body to be 
painted by the painting equipment 

37. A method as in anyone of claims 32 to 36, 
further comprising the steps of: 

(c) providing a visual display (138; 153) of at 
least some of the images of the first scene from 
the camera means which shows at least the first 
portion of the second object and the portion of first 
object if any relatively near to the first portion of 
the second object; and 

(d) providing a symbol (390; 420; 422; 426) 
occupying a predetermined amount of area which 
is illustrative of a guarded zone adjacent to the first 



portion (77; 426) of the second object (80) in to 
which the first object (62) is normally not supposed 
to intrude. 

38. A method as in claim 37, wherein the 
predetermined amount of area may be enlarged in 
order to simulate the response of the image analy- 
sis system and painting equipment to an intrusion 
of a portion of the first object into the guarded 
zone. 
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